[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:50:10 dist-test-slave-9mr0 krb5kdc[18975](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:50:10 dist-test-slave-9mr0 krb5kdc[18975](info): set up 2 sockets
Jun 19 02:50:10 dist-test-slave-9mr0 krb5kdc[18975](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260619 02:50:11.568715 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.966s user 0.007s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:50:11 dist-test-slave-9mr0 krb5kdc[18975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:50:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:50:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:50:11.733042 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:33683
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:43625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:33683
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:12.062844 18988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:12.063198 18988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:12.063308 18988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:12.070823 18988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:50:12.070947 18988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:12.070994 18988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:12.071049 18988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:50:12.071090 18988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:50:12.081480 18988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:43625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:33683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:33683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:12.082844 18988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:12.084116 18988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:12.096282 18993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:12.096239 18996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:12.096247 18994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:12.097059 18988 server_base.cc:1034] running on GCE node
I20260619 02:50:12.097877 18988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:12.099438 18988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:12.100656 18988 hybrid_clock.cc:648] HybridClock initialized: now 1781837412100560 us; error 89 us; skew 500 ppm
I20260619 02:50:12.101015 18988 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:12 dist-test-slave-9mr0 krb5kdc[18975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837412, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:12.104382 18988 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:50:12.105880 18988 webserver.cc:460] Webserver started at http://127.18.134.62:43735/ using document root <none> and password file <none>
I20260619 02:50:12.106582 18988 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:12.106810 18988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:12.107164 18988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:12.110275 18988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "45fe8dc8a90947dd9da99267f2f54445"
format_stamp: "Formatted at 2026-06-19 02:50:12 on dist-test-slave-9mr0"
server_key: "7d9b81fed2b6f3b0f56c918df5afb377"
server_key_iv: "f0fe5a96fffe46d768debee5a33160b1"
server_key_version: "encryptionkey@0"
I20260619 02:50:12.111068 18988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "45fe8dc8a90947dd9da99267f2f54445"
format_stamp: "Formatted at 2026-06-19 02:50:12 on dist-test-slave-9mr0"
server_key: "7d9b81fed2b6f3b0f56c918df5afb377"
server_key_iv: "f0fe5a96fffe46d768debee5a33160b1"
server_key_version: "encryptionkey@0"
I20260619 02:50:12.116473 18988 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 02:50:12.120740 19003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:12.122808 18988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:50:12.122998 18988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "45fe8dc8a90947dd9da99267f2f54445"
format_stamp: "Formatted at 2026-06-19 02:50:12 on dist-test-slave-9mr0"
server_key: "7d9b81fed2b6f3b0f56c918df5afb377"
server_key_iv: "f0fe5a96fffe46d768debee5a33160b1"
server_key_version: "encryptionkey@0"
I20260619 02:50:12.123200 18988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:12.149756 18988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:12.154175 18988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:12.154424 18988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:12.173955 18988 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:33683
I20260619 02:50:12.174003 19054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:33683 every 8 connection(s)
I20260619 02:50:12.175359 18988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:50:12.179388 19055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:12.179491 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 18988
I20260619 02:50:12.179781 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:50:12.181663 18968 external_mini_cluster.cc:1428] Setting key 57b1abd4f89cd99adf46bba7df85995d
I20260619 02:50:12.190729 19055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45fe8dc8a90947dd9da99267f2f54445: Bootstrap starting.
I20260619 02:50:12.194051 19055 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45fe8dc8a90947dd9da99267f2f54445: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:12.195227 19055 log.cc:824] T 00000000000000000000000000000000 P 45fe8dc8a90947dd9da99267f2f54445: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:12.198515 19055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45fe8dc8a90947dd9da99267f2f54445: No bootstrap required, opened a new log
I20260619 02:50:12.206884 19055 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45fe8dc8a90947dd9da99267f2f54445 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45fe8dc8a90947dd9da99267f2f54445" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33683 } }
I20260619 02:50:12.207365 19055 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45fe8dc8a90947dd9da99267f2f54445 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:12.207481 19055 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45fe8dc8a90947dd9da99267f2f54445 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45fe8dc8a90947dd9da99267f2f54445, State: Initialized, Role: FOLLOWER
I20260619 02:50:12.208145 19055 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45fe8dc8a90947dd9da99267f2f54445 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45fe8dc8a90947dd9da99267f2f54445" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33683 } }
I20260619 02:50:12.208382 19055 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45fe8dc8a90947dd9da99267f2f54445 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:50:12.208529 19055 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45fe8dc8a90947dd9da99267f2f54445 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:50:12.208702 19055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45fe8dc8a90947dd9da99267f2f54445 [term 0 FOLLOWER]: Advancing to term 1
Jun 19 02:50:12 dist-test-slave-9mr0 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:12.211452 19055 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45fe8dc8a90947dd9da99267f2f54445 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45fe8dc8a90947dd9da99267f2f54445" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33683 } }
I20260619 02:50:12.212062 19055 leader_election.cc:302] T 00000000000000000000000000000000 P 45fe8dc8a90947dd9da99267f2f54445 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45fe8dc8a90947dd9da99267f2f54445; no voters:
I20260619 02:50:12.212909 19055 leader_election.cc:288] T 00000000000000000000000000000000 P 45fe8dc8a90947dd9da99267f2f54445 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:50:12.213069 19060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45fe8dc8a90947dd9da99267f2f54445 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:50:12.213869 19060 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45fe8dc8a90947dd9da99267f2f54445 [term 1 LEADER]: Becoming Leader. State: Replica: 45fe8dc8a90947dd9da99267f2f54445, State: Running, Role: LEADER
I20260619 02:50:12.214483 19060 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45fe8dc8a90947dd9da99267f2f54445 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45fe8dc8a90947dd9da99267f2f54445" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33683 } }
I20260619 02:50:12.215479 19055 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45fe8dc8a90947dd9da99267f2f54445 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:50:12.218802 19061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45fe8dc8a90947dd9da99267f2f54445 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45fe8dc8a90947dd9da99267f2f54445" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45fe8dc8a90947dd9da99267f2f54445" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33683 } } }
I20260619 02:50:12.219221 19061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45fe8dc8a90947dd9da99267f2f54445 [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:12.223922 19062 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45fe8dc8a90947dd9da99267f2f54445 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45fe8dc8a90947dd9da99267f2f54445. Latest consensus state: current_term: 1 leader_uuid: "45fe8dc8a90947dd9da99267f2f54445" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45fe8dc8a90947dd9da99267f2f54445" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33683 } } }
I20260619 02:50:12.224332 19062 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45fe8dc8a90947dd9da99267f2f54445 [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:12.224481 19065 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:50:12.224880 19057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:12.185844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47312 (local address 127.18.134.62:33683)
0619 02:50:12.186762 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:12.186780 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:12.188039 (+ 1259us) server_negotiation.cc:407] Connection header received
0619 02:50:12.197250 (+ 9211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:12.197275 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:12.197700 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:12.198095 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:12.201268 (+ 3173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:12.202616 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:12.204315 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:12.205020 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:12.205214 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:12.211343 (+ 6129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:12.211352 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:12.211369 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:12.211412 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:12.215702 (+ 4290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:12.217004 (+ 1302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:12.217014 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:12.217024 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:12.217459 (+ 435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:12.218124 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:12.218129 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:12.218132 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:12.218901 (+ 769us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:12.219260 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:12.222249 (+ 2989us) server_negotiation.cc:299] Negotiation successful
0619 02:50:12.223134 (+ 885us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":372,"threads_started":1}
I20260619 02:50:12.236594 19065 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:50:12.246310 19065 catalog_manager.cc:1347] Generated new cluster ID: 1413af630f9d485bacbdf8fa72fdcc49
I20260619 02:50:12.246443 19065 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:50:12.255273 19065 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:50:12.257126 19065 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:50:12.270205 19065 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45fe8dc8a90947dd9da99267f2f54445: Generated new TSK 0
I20260619 02:50:12.271561 19065 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.134.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
I20260619 02:50:12.326984 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:0
--local_ip_for_outbound_sockets=127.18.134.1
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:33683
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:43625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:12.659288 19081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:12.659595 19081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:12.659729 19081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:12.667553 19081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:12.667702 19081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:12.667860 19081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:50:12.677781 19081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:43625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:33683
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:12.679237 19081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:12.680656 19081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:12.694217 19087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:12.694278 19086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:12.694245 19089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:12.694983 19081 server_base.cc:1034] running on GCE node
I20260619 02:50:12.695631 19081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:12.696646 19081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:12.697877 19081 hybrid_clock.cc:648] HybridClock initialized: now 1781837412697827 us; error 47 us; skew 500 ppm
I20260619 02:50:12.698204 19081 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:12 dist-test-slave-9mr0 krb5kdc[18975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837412, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:12.701696 19081 init.cc:377] Logged in from keytab as kudu/127.18.134.1@KRBTEST.COM (short username kudu)
I20260619 02:50:12.703387 19081 webserver.cc:460] Webserver started at http://127.18.134.1:36821/ using document root <none> and password file <none>
I20260619 02:50:12.704097 19081 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:12.704213 19081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:12.704519 19081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:12.707639 19081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-0/data/instance:
uuid: "c3496aee6b07423b84054e265e925f4c"
format_stamp: "Formatted at 2026-06-19 02:50:12 on dist-test-slave-9mr0"
server_key: "54e2a05819f69630e87233b50d4abeb7"
server_key_iv: "0d2be54de8a79b37a90791227470970d"
server_key_version: "encryptionkey@0"
I20260619 02:50:12.708426 19081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance:
uuid: "c3496aee6b07423b84054e265e925f4c"
format_stamp: "Formatted at 2026-06-19 02:50:12 on dist-test-slave-9mr0"
server_key: "54e2a05819f69630e87233b50d4abeb7"
server_key_iv: "0d2be54de8a79b37a90791227470970d"
server_key_version: "encryptionkey@0"
I20260619 02:50:12.713970 19081 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 02:50:12.717765 19096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:12.719264 19081 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:50:12.719432 19081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "c3496aee6b07423b84054e265e925f4c"
format_stamp: "Formatted at 2026-06-19 02:50:12 on dist-test-slave-9mr0"
server_key: "54e2a05819f69630e87233b50d4abeb7"
server_key_iv: "0d2be54de8a79b37a90791227470970d"
server_key_version: "encryptionkey@0"
I20260619 02:50:12.719633 19081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:12.746304 19081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:12.750824 19081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:12.751101 19081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:12.752135 19081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:50:12.753521 19081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:12.753599 19081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:12.753719 19081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:12.753773 19081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:12.787854 19081 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:38329
I20260619 02:50:12.787906 19208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:38329 every 8 connection(s)
I20260619 02:50:12.789151 19081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:50:12.794269 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 19081
I20260619 02:50:12.794464 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance
I20260619 02:50:12.795284 18968 external_mini_cluster.cc:1428] Setting key 7ec88a7233dcbc1ac258199f2760949d
Jun 19 02:50:12 dist-test-slave-9mr0 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837412, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.134.2@KRBTEST.COM; defaulting to no policy
I20260619 02:50:12.812342 19211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:12.792002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:58723 (local address 127.18.134.62:33683)
0619 02:50:12.792509 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:12.792515 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:12.793690 (+ 1175us) server_negotiation.cc:407] Connection header received
0619 02:50:12.796235 (+ 2545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:12.796241 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:12.796317 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:12.796421 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:12.798183 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:12.799343 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:12.801575 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:12.801989 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:12.802105 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:12.805741 (+ 3636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:12.805748 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:12.805753 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:12.805794 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:12.809004 (+ 3210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:12.809857 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:12.809863 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:12.809867 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:12.810014 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:12.810569 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:12.810574 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:12.810578 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:12.810967 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:12.811114 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:12.811914 (+ 800us) server_negotiation.cc:299] Negotiation successful
0619 02:50:12.812084 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":260,"threads_started":1}
I20260619 02:50:12.813396 19212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:12.792568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33683 (local address 127.18.134.1:58723)
0619 02:50:12.793490 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:12.793540 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:12.795911 (+ 2371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:12.796739 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:12.796753 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:12.797387 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:12.798001 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:12.798019 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:12.799566 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:12.799571 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:12.801407 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:12.801425 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:12.802219 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:12.802226 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:12.802476 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:12.803166 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:12.803189 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:12.805480 (+ 2291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:12.809217 (+ 3737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:12.809227 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:12.809243 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:12.809724 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:12.810194 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:12.810198 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:12.810200 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:12.810446 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:12.811108 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:12.811115 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:12.811478 (+ 363us) client_negotiation.cc:769] Sending connection context
0619 02:50:12.811752 (+ 274us) client_negotiation.cc:240] Negotiation successful
0619 02:50:12.812218 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":474,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.2@KRBTEST.COM" created.
I20260619 02:50:12.815260 19209 heartbeater.cc:347] Connected to a master server at 127.18.134.62:33683
I20260619 02:50:12.815587 19209 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:12.816493 19209 heartbeater.cc:511] Master 127.18.134.62:33683 requested a full tablet report, sending...
I20260619 02:50:12.818856 19020 ts_manager.cc:194] Registered new tserver with Master: c3496aee6b07423b84054e265e925f4c (127.18.134.1:38329)
I20260619 02:50:12.820307 19020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.1@KRBTEST.COM'} at 127.18.134.1:58723
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
I20260619 02:50:12.838160 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:0
--local_ip_for_outbound_sockets=127.18.134.2
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:33683
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:43625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:13.171916 19215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:13.172220 19215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:13.172317 19215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:13.179728 19215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:13.179862 19215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:13.180014 19215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:50:13.190268 19215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:43625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:33683
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:13.191589 19215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:13.192867 19215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:13.206404 19223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:13.206423 19221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:13.206750 19220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:13.207620 19215 server_base.cc:1034] running on GCE node
I20260619 02:50:13.208267 19215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:13.209265 19215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:13.210480 19215 hybrid_clock.cc:648] HybridClock initialized: now 1781837413210428 us; error 49 us; skew 500 ppm
I20260619 02:50:13.210853 19215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:13 dist-test-slave-9mr0 krb5kdc[18975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837413, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:13.213968 19215 init.cc:377] Logged in from keytab as kudu/127.18.134.2@KRBTEST.COM (short username kudu)
I20260619 02:50:13.215813 19215 webserver.cc:460] Webserver started at http://127.18.134.2:38439/ using document root <none> and password file <none>
I20260619 02:50:13.216557 19215 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:13.216677 19215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:13.216986 19215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:13.220019 19215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-1/data/instance:
uuid: "76763317413f4420904f28c6caeead59"
format_stamp: "Formatted at 2026-06-19 02:50:13 on dist-test-slave-9mr0"
server_key: "7a9d33fa2bab03c9c7b78595529c8709"
server_key_iv: "69ba8e6bf7547a82de70bab3134e2bee"
server_key_version: "encryptionkey@0"
I20260619 02:50:13.220799 19215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance:
uuid: "76763317413f4420904f28c6caeead59"
format_stamp: "Formatted at 2026-06-19 02:50:13 on dist-test-slave-9mr0"
server_key: "7a9d33fa2bab03c9c7b78595529c8709"
server_key_iv: "69ba8e6bf7547a82de70bab3134e2bee"
server_key_version: "encryptionkey@0"
I20260619 02:50:13.226522 19215 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260619 02:50:13.230540 19230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:13.232100 19215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:50:13.232316 19215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "76763317413f4420904f28c6caeead59"
format_stamp: "Formatted at 2026-06-19 02:50:13 on dist-test-slave-9mr0"
server_key: "7a9d33fa2bab03c9c7b78595529c8709"
server_key_iv: "69ba8e6bf7547a82de70bab3134e2bee"
server_key_version: "encryptionkey@0"
I20260619 02:50:13.232542 19215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:13.253656 19215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:13.257863 19215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:13.258131 19215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:13.259148 19215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:50:13.260490 19215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:13.260560 19215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:13.260654 19215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:13.260708 19215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:13.295778 19215 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:43303
I20260619 02:50:13.295830 19342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:43303 every 8 connection(s)
I20260619 02:50:13.297077 19215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:50:13.304078 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 19215
I20260619 02:50:13.304255 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance
I20260619 02:50:13.304684 18968 external_mini_cluster.cc:1428] Setting key 50b719d0018129e3ed9dafbf78b6ad23
Jun 19 02:50:13 dist-test-slave-9mr0 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837413, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.134.3@KRBTEST.COM; defaulting to no policy
I20260619 02:50:13.318514 19211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:13.299708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:50915 (local address 127.18.134.62:33683)
0619 02:50:13.299923 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:13.299928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:13.301133 (+ 1205us) server_negotiation.cc:407] Connection header received
0619 02:50:13.302976 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:13.302981 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:13.303053 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:13.303145 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:13.304802 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:13.305548 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:13.307632 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:13.308308 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:13.308452 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:13.312081 (+ 3629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:13.312086 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:13.312090 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:13.312127 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:13.315336 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:13.316135 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:13.316140 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:13.316144 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:13.316304 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:13.316742 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:13.316748 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:13.316751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:13.317019 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:13.317150 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:13.317943 (+ 793us) server_negotiation.cc:299] Negotiation successful
0619 02:50:13.318183 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260619 02:50:13.319670 19345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:13.300204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33683 (local address 127.18.134.2:50915)
0619 02:50:13.300962 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:13.301001 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:13.302693 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:13.303352 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:13.303361 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:13.303941 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:13.304613 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:13.304631 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:13.305753 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:13.305759 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:13.307461 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:13.307479 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:13.308541 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:13.308546 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:13.308782 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:13.309462 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:13.309487 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:13.311808 (+ 2321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:13.315506 (+ 3698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:13.315518 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:13.315535 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:13.315990 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:13.316412 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:13.316417 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:13.316421 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:13.316621 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:13.317161 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:13.317170 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:13.317557 (+ 387us) client_negotiation.cc:769] Sending connection context
0619 02:50:13.317819 (+ 262us) client_negotiation.cc:240] Negotiation successful
0619 02:50:13.318399 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":276,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.3@KRBTEST.COM" created.
I20260619 02:50:13.321383 19343 heartbeater.cc:347] Connected to a master server at 127.18.134.62:33683
I20260619 02:50:13.321765 19343 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:13.322714 19343 heartbeater.cc:511] Master 127.18.134.62:33683 requested a full tablet report, sending...
I20260619 02:50:13.324467 19020 ts_manager.cc:194] Registered new tserver with Master: 76763317413f4420904f28c6caeead59 (127.18.134.2:43303)
I20260619 02:50:13.325400 19020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.2@KRBTEST.COM'} at 127.18.134.2:50915
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
I20260619 02:50:13.342049 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:0
--local_ip_for_outbound_sockets=127.18.134.3
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:33683
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:43625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:13.692510 19348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:13.692809 19348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:13.692910 19348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:13.700667 19348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:13.700798 19348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:13.700914 19348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:50:13.711366 19348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:43625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:33683
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:13.712685 19348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:13.713994 19348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:13.727666 19353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:13.727716 19356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:13.727633 19354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:13.728703 19348 server_base.cc:1034] running on GCE node
I20260619 02:50:13.729303 19348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:13.730413 19348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:13.731721 19348 hybrid_clock.cc:648] HybridClock initialized: now 1781837413731615 us; error 111 us; skew 500 ppm
I20260619 02:50:13.732079 19348 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:13 dist-test-slave-9mr0 krb5kdc[18975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837413, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:13.735303 19348 init.cc:377] Logged in from keytab as kudu/127.18.134.3@KRBTEST.COM (short username kudu)
I20260619 02:50:13.736864 19348 webserver.cc:460] Webserver started at http://127.18.134.3:40169/ using document root <none> and password file <none>
I20260619 02:50:13.737540 19348 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:13.737650 19348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:13.737949 19348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:13.741734 19348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-2/data/instance:
uuid: "53bec16c06fb4ac7b0c01c0822bccdae"
format_stamp: "Formatted at 2026-06-19 02:50:13 on dist-test-slave-9mr0"
server_key: "0c367f192d51e2af35d2960972f5dd00"
server_key_iv: "138588271232347319618e6aae645903"
server_key_version: "encryptionkey@0"
I20260619 02:50:13.742487 19348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance:
uuid: "53bec16c06fb4ac7b0c01c0822bccdae"
format_stamp: "Formatted at 2026-06-19 02:50:13 on dist-test-slave-9mr0"
server_key: "0c367f192d51e2af35d2960972f5dd00"
server_key_iv: "138588271232347319618e6aae645903"
server_key_version: "encryptionkey@0"
I20260619 02:50:13.747916 19348 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 02:50:13.751461 19363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:13.753007 19348 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:50:13.753175 19348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "53bec16c06fb4ac7b0c01c0822bccdae"
format_stamp: "Formatted at 2026-06-19 02:50:13 on dist-test-slave-9mr0"
server_key: "0c367f192d51e2af35d2960972f5dd00"
server_key_iv: "138588271232347319618e6aae645903"
server_key_version: "encryptionkey@0"
I20260619 02:50:13.753351 19348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:13.772895 19348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:13.777364 19348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:13.777616 19348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:13.778759 19348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:50:13.780185 19348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:13.780262 19348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:13.780774 19348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:13.780821 19348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:13.817430 19348 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:45759
I20260619 02:50:13.817466 19475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:45759 every 8 connection(s)
I20260619 02:50:13.818833 19348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:50:13.823751 19209 heartbeater.cc:503] Master 127.18.134.62:33683 was elected leader, sending a full tablet report...
I20260619 02:50:13.828913 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 19348
I20260619 02:50:13.829110 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance
I20260619 02:50:13.829622 18968 external_mini_cluster.cc:1428] Setting key 261c5533077bc8851ff8bc2358dff72a
Jun 19 02:50:13 dist-test-slave-9mr0 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837413, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:13.840735 19478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:13.821413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:43009 (local address 127.18.134.62:33683)
0619 02:50:13.821856 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:13.821863 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:13.822887 (+ 1024us) server_negotiation.cc:407] Connection header received
0619 02:50:13.824926 (+ 2039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:13.824931 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:13.825014 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:13.825105 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:13.826921 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:13.827745 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:13.829798 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:13.830319 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:13.830502 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:13.834894 (+ 4392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:13.834907 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:13.834910 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:13.834970 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:13.837671 (+ 2701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:13.838460 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:13.838465 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:13.838467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:13.838607 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:13.839087 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:13.839091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:13.839093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:13.839402 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:13.839520 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:13.840265 (+ 745us) server_negotiation.cc:299] Negotiation successful
0619 02:50:13.840446 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":215,"threads_started":1}
I20260619 02:50:13.841660 19479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:13.821980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33683 (local address 127.18.134.3:43009)
0619 02:50:13.822715 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:13.822777 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:13.824618 (+ 1841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:13.825371 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:13.825383 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:13.826082 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:13.826749 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:13.826769 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:13.827895 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:13.827900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:13.829524 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:13.829539 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:13.830615 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:13.830621 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:13.830886 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:13.832204 (+ 1318us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:13.832243 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:13.834605 (+ 2362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:13.837837 (+ 3232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:13.837847 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:13.837868 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:13.838328 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:13.838744 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:13.838748 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:13.838751 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:13.838957 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:13.839521 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:13.839528 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:13.839883 (+ 355us) client_negotiation.cc:769] Sending connection context
0619 02:50:13.840115 (+ 232us) client_negotiation.cc:240] Negotiation successful
0619 02:50:13.840540 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":275,"threads_started":1}
I20260619 02:50:13.843197 19476 heartbeater.cc:347] Connected to a master server at 127.18.134.62:33683
I20260619 02:50:13.843494 19476 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:13.844231 19476 heartbeater.cc:511] Master 127.18.134.62:33683 requested a full tablet report, sending...
I20260619 02:50:13.846016 19020 ts_manager.cc:194] Registered new tserver with Master: 53bec16c06fb4ac7b0c01c0822bccdae (127.18.134.3:45759)
I20260619 02:50:13.847010 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:50:13.847011 19020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.3@KRBTEST.COM'} at 127.18.134.3:43009
Jun 19 02:50:13 dist-test-slave-9mr0 krb5kdc[18975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837413, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 02:50:13 dist-test-slave-9mr0 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837413, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.134.1@KRBTEST.COM
I20260619 02:50:13.889819 19484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:13.874007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56020 (local address 127.18.134.1:38329)
0619 02:50:13.874716 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:13.874726 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:13.874760 (+ 34us) server_negotiation.cc:407] Connection header received
0619 02:50:13.874840 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:13.874851 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:13.875062 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:13.875229 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:13.877030 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:13.878037 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:13.879385 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:13.879990 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:13.880106 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:13.884094 (+ 3988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:13.884108 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:13.884123 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:13.884163 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:13.887064 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:13.887770 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:13.887777 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:13.887779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:13.887909 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:13.888299 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:13.888302 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:13.888304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:13.888640 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:13.888912 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:13.889248 (+ 336us) server_negotiation.cc:299] Negotiation successful
0619 02:50:13.889495 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":319,"threads_started":1}
W20260619 02:50:13.891139 19124 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56020
Jun 19 02:50:13 dist-test-slave-9mr0 krb5kdc[18975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 02:50:13 dist-test-slave-9mr0 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.1@KRBTEST.COM
I20260619 02:50:13.933061 19484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:13.918919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56024 (local address 127.18.134.1:38329)
0619 02:50:13.919144 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:13.919150 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:13.919471 (+ 321us) server_negotiation.cc:407] Connection header received
0619 02:50:13.919643 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:13.919649 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:13.919755 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:13.919853 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:13.921169 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:13.921851 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:13.923087 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:13.923497 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:13.923607 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:13.927791 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:13.927797 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:13.927800 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:13.927839 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:13.930311 (+ 2472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:13.931097 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:13.931102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:13.931104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:13.931259 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:13.931627 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:13.931630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:13.931632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:13.931902 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:13.932077 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:13.932637 (+ 560us) server_negotiation.cc:299] Negotiation successful
0619 02:50:13.932831 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 02:50:13.934788 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 19081
I20260619 02:50:13.948030 19484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:13.935765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56026 (local address 127.18.134.1:38329)
0619 02:50:13.935948 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:13.935952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:13.936284 (+ 332us) server_negotiation.cc:407] Connection header received
0619 02:50:13.936391 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:13.936394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:13.936453 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:13.936530 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:13.937689 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:13.938407 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:13.939699 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:13.940152 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:13.940290 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:13.942908 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:13.942913 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:13.942916 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:13.942949 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:13.945509 (+ 2560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:13.946194 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:13.946198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:13.946200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:13.946352 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:13.946758 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:13.946762 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:13.946764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:13.947016 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:13.947187 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:13.947573 (+ 386us) server_negotiation.cc:299] Negotiation successful
0619 02:50:13.947755 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 02:50:13.948732 19204 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:14.031132 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 19081
I20260619 02:50:14.050096 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 19215
Jun 19 02:50:14 dist-test-slave-9mr0 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.2@KRBTEST.COM
I20260619 02:50:14.066161 19491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:14.050979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60510 (local address 127.18.134.2:43303)
0619 02:50:14.051481 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:14.051489 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:14.051517 (+ 28us) server_negotiation.cc:407] Connection header received
0619 02:50:14.051576 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:14.051582 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:14.051795 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:14.051929 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:14.052991 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:14.053987 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:14.055497 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:14.056046 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:14.056165 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:14.060185 (+ 4020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:14.060198 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:14.060218 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:14.060262 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:14.063130 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:14.063872 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:14.063880 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:14.063883 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:14.064021 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:14.064468 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:14.064472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:14.064474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:14.064852 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:14.065158 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:14.065500 (+ 342us) server_negotiation.cc:299] Negotiation successful
0619 02:50:14.065781 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":247,"threads_started":1}
I20260619 02:50:14.067589 19338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:14.147975 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 19215
I20260619 02:50:14.160887 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 19348
Jun 19 02:50:14 dist-test-slave-9mr0 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.3@KRBTEST.COM
I20260619 02:50:14.177697 19493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:14.161924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48384 (local address 127.18.134.3:45759)
0619 02:50:14.162502 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:14.162509 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:14.162538 (+ 29us) server_negotiation.cc:407] Connection header received
0619 02:50:14.162597 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:14.162603 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:14.162873 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:14.163002 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:14.164246 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:14.165320 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:14.166617 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:14.167261 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:14.167363 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:14.171508 (+ 4145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:14.171521 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:14.171536 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:14.171577 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:14.174715 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:14.175446 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:14.175455 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:14.175459 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:14.175597 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:14.175899 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:14.175903 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:14.175907 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:14.176304 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:14.176577 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:14.176816 (+ 239us) server_negotiation.cc:299] Negotiation successful
0619 02:50:14.177065 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":308,"threads_started":1}
I20260619 02:50:14.179024 19471 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:14.259958 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 19348
I20260619 02:50:14.272145 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 18988
I20260619 02:50:14.273573 19050 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:14.373752 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 18988
2026-06-19T02:50:14Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3817 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:50:14 dist-test-slave-9mr0 krb5kdc[19498](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:50:14 dist-test-slave-9mr0 krb5kdc[19498](info): set up 2 sockets
Jun 19 02:50:14 dist-test-slave-9mr0 krb5kdc[19498](info): commencing operation
krb5kdc: starting...
W20260619 02:50:15.370255 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.959s user 0.004s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:50:15 dist-test-slave-9mr0 krb5kdc[19498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:50:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:50:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:50:15.529783 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:45621
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:46287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:45621
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:15.828100 19511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:15.828389 19511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:15.828507 19511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:15.835497 19511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:50:15.835609 19511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:15.835687 19511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:15.835767 19511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:50:15.835835 19511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:50:15.845745 19511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:46287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:45621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:45621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:15.847057 19511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:15.848348 19511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:15.859157 19516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:15.859149 19517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:15.859166 19519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:15.860193 19511 server_base.cc:1034] running on GCE node
I20260619 02:50:15.860893 19511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:15.862164 19511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:15.863359 19511 hybrid_clock.cc:648] HybridClock initialized: now 1781837415863308 us; error 40 us; skew 500 ppm
I20260619 02:50:15.863663 19511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:15 dist-test-slave-9mr0 krb5kdc[19498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837415, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:15.871585 19511 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:50:15.873077 19511 webserver.cc:460] Webserver started at http://127.18.134.62:35243/ using document root <none> and password file <none>
I20260619 02:50:15.873736 19511 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:15.873847 19511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:15.874130 19511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:15.876960 19511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "ebc88c02f64a41afb2552c9681033cc2"
format_stamp: "Formatted at 2026-06-19 02:50:15 on dist-test-slave-9mr0"
server_key: "508e1dc5e117bf86a8855cb6326ed625"
server_key_iv: "0eef641fd1381ccc568d22f4a3009902"
server_key_version: "encryptionkey@0"
I20260619 02:50:15.877645 19511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "ebc88c02f64a41afb2552c9681033cc2"
format_stamp: "Formatted at 2026-06-19 02:50:15 on dist-test-slave-9mr0"
server_key: "508e1dc5e117bf86a8855cb6326ed625"
server_key_iv: "0eef641fd1381ccc568d22f4a3009902"
server_key_version: "encryptionkey@0"
I20260619 02:50:15.882376 19511 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260619 02:50:15.885605 19526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:15.886904 19511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:50:15.887045 19511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "ebc88c02f64a41afb2552c9681033cc2"
format_stamp: "Formatted at 2026-06-19 02:50:15 on dist-test-slave-9mr0"
server_key: "508e1dc5e117bf86a8855cb6326ed625"
server_key_iv: "0eef641fd1381ccc568d22f4a3009902"
server_key_version: "encryptionkey@0"
I20260619 02:50:15.887204 19511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:15.900136 19511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:15.903926 19511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:15.904168 19511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:15.920730 19511 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:45621
I20260619 02:50:15.920750 19577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:45621 every 8 connection(s)
I20260619 02:50:15.922030 19511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:50:15.923200 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 19511
I20260619 02:50:15.923414 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:50:15.924201 18968 external_mini_cluster.cc:1428] Setting key 7aa437efcb3d95ac82af769c1844fc0f
I20260619 02:50:15.926519 19578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 19 02:50:15 dist-test-slave-9mr0 krb5kdc[19498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:15.936815 19578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebc88c02f64a41afb2552c9681033cc2: Bootstrap starting.
I20260619 02:50:15.940114 19578 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebc88c02f64a41afb2552c9681033cc2: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:15.941111 19578 log.cc:824] T 00000000000000000000000000000000 P ebc88c02f64a41afb2552c9681033cc2: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:15.944164 19578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebc88c02f64a41afb2552c9681033cc2: No bootstrap required, opened a new log
I20260619 02:50:15.944890 19581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:15.926184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52654 (local address 127.18.134.62:45621)
0619 02:50:15.926941 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:15.926953 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:15.926992 (+ 39us) server_negotiation.cc:407] Connection header received
0619 02:50:15.928531 (+ 1539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:15.928552 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:15.928939 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:15.929293 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:15.930696 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:15.931970 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:15.933125 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:15.933781 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:15.933959 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:15.937861 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:15.937869 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:15.937883 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:15.937925 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:15.940858 (+ 2933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:15.941527 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:15.941536 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:15.941544 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:15.941769 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:15.942141 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:15.942146 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:15.942149 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:15.942734 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:15.942939 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:15.943483 (+ 544us) server_negotiation.cc:299] Negotiation successful
0619 02:50:15.943819 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":335,"threads_started":1}
I20260619 02:50:15.951287 19578 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebc88c02f64a41afb2552c9681033cc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebc88c02f64a41afb2552c9681033cc2" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 45621 } }
I20260619 02:50:15.951654 19578 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebc88c02f64a41afb2552c9681033cc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:15.951737 19578 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebc88c02f64a41afb2552c9681033cc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebc88c02f64a41afb2552c9681033cc2, State: Initialized, Role: FOLLOWER
I20260619 02:50:15.952239 19578 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebc88c02f64a41afb2552c9681033cc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebc88c02f64a41afb2552c9681033cc2" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 45621 } }
I20260619 02:50:15.952468 19578 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebc88c02f64a41afb2552c9681033cc2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:50:15.952582 19578 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebc88c02f64a41afb2552c9681033cc2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:50:15.952721 19578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebc88c02f64a41afb2552c9681033cc2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:15.954814 19578 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebc88c02f64a41afb2552c9681033cc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebc88c02f64a41afb2552c9681033cc2" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 45621 } }
I20260619 02:50:15.955289 19578 leader_election.cc:302] T 00000000000000000000000000000000 P ebc88c02f64a41afb2552c9681033cc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebc88c02f64a41afb2552c9681033cc2; no voters:
I20260619 02:50:15.955863 19578 leader_election.cc:288] T 00000000000000000000000000000000 P ebc88c02f64a41afb2552c9681033cc2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:50:15.955991 19583 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebc88c02f64a41afb2552c9681033cc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:50:15.956703 19583 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebc88c02f64a41afb2552c9681033cc2 [term 1 LEADER]: Becoming Leader. State: Replica: ebc88c02f64a41afb2552c9681033cc2, State: Running, Role: LEADER
I20260619 02:50:15.957180 19583 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebc88c02f64a41afb2552c9681033cc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebc88c02f64a41afb2552c9681033cc2" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 45621 } }
I20260619 02:50:15.957803 19578 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebc88c02f64a41afb2552c9681033cc2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:50:15.959966 19584 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebc88c02f64a41afb2552c9681033cc2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebc88c02f64a41afb2552c9681033cc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebc88c02f64a41afb2552c9681033cc2" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 45621 } } }
I20260619 02:50:15.959973 19585 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebc88c02f64a41afb2552c9681033cc2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebc88c02f64a41afb2552c9681033cc2. Latest consensus state: current_term: 1 leader_uuid: "ebc88c02f64a41afb2552c9681033cc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebc88c02f64a41afb2552c9681033cc2" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 45621 } } }
I20260619 02:50:15.960482 19585 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebc88c02f64a41afb2552c9681033cc2 [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:15.961215 19584 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebc88c02f64a41afb2552c9681033cc2 [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:15.962373 19591 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:50:15.966831 19591 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:50:15.974860 19591 catalog_manager.cc:1347] Generated new cluster ID: da8dfd2d3ee44569945757f309cb227f
I20260619 02:50:15.974973 19591 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:50:15.999377 19591 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:50:16.000675 19591 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:50:16.011586 19591 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebc88c02f64a41afb2552c9681033cc2: Generated new TSK 0
I20260619 02:50:16.012482 19591 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.134.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
I20260619 02:50:16.058197 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:0
--local_ip_for_outbound_sockets=127.18.134.1
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:45621
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:46287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:16.368988 19604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:16.369263 19604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:16.369381 19604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:16.376562 19604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:16.376672 19604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:16.376825 19604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:50:16.386555 19604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:46287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:45621
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:16.387878 19604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:16.389220 19604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:16.402168 19612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:16.402289 19609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:16.402252 19610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:16.403060 19604 server_base.cc:1034] running on GCE node
I20260619 02:50:16.403646 19604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:16.404598 19604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:16.405826 19604 hybrid_clock.cc:648] HybridClock initialized: now 1781837416405753 us; error 55 us; skew 500 ppm
I20260619 02:50:16.406152 19604 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:16 dist-test-slave-9mr0 krb5kdc[19498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837416, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:16.409293 19604 init.cc:377] Logged in from keytab as kudu/127.18.134.1@KRBTEST.COM (short username kudu)
I20260619 02:50:16.410912 19604 webserver.cc:460] Webserver started at http://127.18.134.1:43937/ using document root <none> and password file <none>
I20260619 02:50:16.411585 19604 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:16.411685 19604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:16.411983 19604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:16.414906 19604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-0/data/instance:
uuid: "d02b24949e184a769e2616739436414b"
format_stamp: "Formatted at 2026-06-19 02:50:16 on dist-test-slave-9mr0"
server_key: "57e58366b9ba8b0de47fc9731f255c0a"
server_key_iv: "315d076030e165cf9c744af89021098b"
server_key_version: "encryptionkey@0"
I20260619 02:50:16.415657 19604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance:
uuid: "d02b24949e184a769e2616739436414b"
format_stamp: "Formatted at 2026-06-19 02:50:16 on dist-test-slave-9mr0"
server_key: "57e58366b9ba8b0de47fc9731f255c0a"
server_key_iv: "315d076030e165cf9c744af89021098b"
server_key_version: "encryptionkey@0"
I20260619 02:50:16.420747 19604 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:50:16.424243 19619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:16.425560 19604 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 02:50:16.425700 19604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "d02b24949e184a769e2616739436414b"
format_stamp: "Formatted at 2026-06-19 02:50:16 on dist-test-slave-9mr0"
server_key: "57e58366b9ba8b0de47fc9731f255c0a"
server_key_iv: "315d076030e165cf9c744af89021098b"
server_key_version: "encryptionkey@0"
I20260619 02:50:16.425894 19604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:16.448854 19604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:16.452939 19604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:16.453199 19604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:16.454138 19604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:50:16.455472 19604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:16.455545 19604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:16.455657 19604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:16.455704 19604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:16.485427 19604 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:45389
I20260619 02:50:16.485499 19731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:45389 every 8 connection(s)
I20260619 02:50:16.486681 19604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:50:16.494472 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 19604
I20260619 02:50:16.494679 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance
I20260619 02:50:16.495235 18968 external_mini_cluster.cc:1428] Setting key 7dcfa94c9390a127ce55e359350f7620
Jun 19 02:50:16 dist-test-slave-9mr0 krb5kdc[19498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837416, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:16.505694 19734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:16.489050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:60985 (local address 127.18.134.62:45621)
0619 02:50:16.489476 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:16.489483 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:16.490494 (+ 1011us) server_negotiation.cc:407] Connection header received
0619 02:50:16.492316 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:16.492322 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:16.492389 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:16.492479 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:16.493927 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:16.494688 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:16.496321 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:16.496721 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:16.496833 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:16.500278 (+ 3445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:16.500282 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:16.500286 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:16.500315 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:16.502817 (+ 2502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:16.503548 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:16.503554 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:16.503557 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:16.503666 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:16.504057 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:16.504062 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:16.504065 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:16.504337 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:16.504460 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:16.505129 (+ 669us) server_negotiation.cc:299] Negotiation successful
0619 02:50:16.505349 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":204,"threads_started":1}
I20260619 02:50:16.506750 19735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:16.489616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:45621 (local address 127.18.134.1:60985)
0619 02:50:16.490307 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:16.490394 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:16.492082 (+ 1688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:16.492667 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:16.492675 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:16.493172 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:16.493763 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:16.493785 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:16.494830 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:16.494834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:16.496198 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:16.496210 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:16.496923 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:16.496931 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:16.497150 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:16.497806 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:16.497829 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:16.500058 (+ 2229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:16.502951 (+ 2893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:16.502960 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:16.502979 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:16.503438 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:16.503763 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:16.503768 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:16.503773 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:16.503957 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:16.504452 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:16.504460 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:16.504791 (+ 331us) client_negotiation.cc:769] Sending connection context
0619 02:50:16.505040 (+ 249us) client_negotiation.cc:240] Negotiation successful
0619 02:50:16.505563 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":264,"threads_started":1}
I20260619 02:50:16.508369 19732 heartbeater.cc:347] Connected to a master server at 127.18.134.62:45621
WARNING: no policy specified for kudu/127.18.134.2@KRBTEST.COM; defaulting to no policy
I20260619 02:50:16.508647 19732 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:16.509325 19732 heartbeater.cc:511] Master 127.18.134.62:45621 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.2@KRBTEST.COM" created.
I20260619 02:50:16.511694 19543 ts_manager.cc:194] Registered new tserver with Master: d02b24949e184a769e2616739436414b (127.18.134.1:45389)
I20260619 02:50:16.513139 19543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.1@KRBTEST.COM'} at 127.18.134.1:60985
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
I20260619 02:50:16.530181 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:0
--local_ip_for_outbound_sockets=127.18.134.2
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:45621
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:46287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:16.840363 19738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:16.840652 19738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:16.840750 19738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:16.848026 19738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:16.848140 19738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:16.848253 19738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:50:16.858268 19738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:46287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:45621
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:16.859586 19738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:16.860879 19738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:16.873654 19743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:16.873673 19744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:16.873683 19746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:16.874359 19738 server_base.cc:1034] running on GCE node
I20260619 02:50:16.874989 19738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:16.875929 19738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:16.877168 19738 hybrid_clock.cc:648] HybridClock initialized: now 1781837416877094 us; error 58 us; skew 500 ppm
I20260619 02:50:16.877514 19738 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:16 dist-test-slave-9mr0 krb5kdc[19498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837416, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:16.880746 19738 init.cc:377] Logged in from keytab as kudu/127.18.134.2@KRBTEST.COM (short username kudu)
I20260619 02:50:16.882284 19738 webserver.cc:460] Webserver started at http://127.18.134.2:42299/ using document root <none> and password file <none>
I20260619 02:50:16.883030 19738 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:16.883150 19738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:16.883443 19738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:16.886394 19738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-1/data/instance:
uuid: "a65a64e51b474735bc40e88329c1a469"
format_stamp: "Formatted at 2026-06-19 02:50:16 on dist-test-slave-9mr0"
server_key: "28aa663ec68c653a9f107ea20ef55254"
server_key_iv: "182cf59e107fc8495a59f7ff17c15ba5"
server_key_version: "encryptionkey@0"
I20260619 02:50:16.887234 19738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance:
uuid: "a65a64e51b474735bc40e88329c1a469"
format_stamp: "Formatted at 2026-06-19 02:50:16 on dist-test-slave-9mr0"
server_key: "28aa663ec68c653a9f107ea20ef55254"
server_key_iv: "182cf59e107fc8495a59f7ff17c15ba5"
server_key_version: "encryptionkey@0"
I20260619 02:50:16.892688 19738 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 02:50:16.896512 19753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:16.897897 19738 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 02:50:16.898049 19738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "a65a64e51b474735bc40e88329c1a469"
format_stamp: "Formatted at 2026-06-19 02:50:16 on dist-test-slave-9mr0"
server_key: "28aa663ec68c653a9f107ea20ef55254"
server_key_iv: "182cf59e107fc8495a59f7ff17c15ba5"
server_key_version: "encryptionkey@0"
I20260619 02:50:16.898241 19738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:16.921900 19738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:16.926440 19738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:16.926751 19738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:16.927809 19738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:50:16.929314 19738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:16.929425 19738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:16.929556 19738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:16.929610 19738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:16.960171 19738 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:39057
I20260619 02:50:16.960240 19865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:39057 every 8 connection(s)
I20260619 02:50:16.961408 19738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:50:16.965611 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 19738
I20260619 02:50:16.965788 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance
I20260619 02:50:16.966200 18968 external_mini_cluster.cc:1428] Setting key 02804c14eca64f10b53a548824df787e
Jun 19 02:50:16 dist-test-slave-9mr0 krb5kdc[19498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837416, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.134.3@KRBTEST.COM; defaulting to no policy
I20260619 02:50:16.983047 19734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:16.964013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:57481 (local address 127.18.134.62:45621)
0619 02:50:16.964210 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:16.964215 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:16.965423 (+ 1208us) server_negotiation.cc:407] Connection header received
0619 02:50:16.967481 (+ 2058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:16.967486 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:16.967567 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:16.967650 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:16.969243 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:16.970414 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:16.972235 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:16.972662 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:16.972755 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:16.977160 (+ 4405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:16.977166 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:16.977170 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:16.977206 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:16.979771 (+ 2565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:16.980735 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:16.980739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:16.980741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:16.980888 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:16.981372 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:16.981375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:16.981377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:16.981633 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:16.981781 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:16.982558 (+ 777us) server_negotiation.cc:299] Negotiation successful
0619 02:50:16.982819 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 02:50:16.984020 19868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:16.964498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:45621 (local address 127.18.134.2:57481)
0619 02:50:16.965241 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:16.965289 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:16.967179 (+ 1890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:16.967926 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:16.967939 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:16.968492 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:16.969072 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:16.969089 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:16.970597 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:16.970601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:16.972065 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:16.972078 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:16.972881 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:16.972889 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:16.973360 (+ 471us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:16.974083 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:16.974112 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:16.976872 (+ 2760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:16.979944 (+ 3072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:16.979955 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:16.979976 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:16.980610 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:16.980993 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:16.980998 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:16.981002 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:16.981267 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:16.981763 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:16.981772 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:16.982199 (+ 427us) client_negotiation.cc:769] Sending connection context
0619 02:50:16.982432 (+ 233us) client_negotiation.cc:240] Negotiation successful
0619 02:50:16.982891 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":274,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.3@KRBTEST.COM" created.
I20260619 02:50:16.985693 19866 heartbeater.cc:347] Connected to a master server at 127.18.134.62:45621
I20260619 02:50:16.986042 19866 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:16.986817 19866 heartbeater.cc:511] Master 127.18.134.62:45621 requested a full tablet report, sending...
I20260619 02:50:16.988265 19543 ts_manager.cc:194] Registered new tserver with Master: a65a64e51b474735bc40e88329c1a469 (127.18.134.2:39057)
I20260619 02:50:16.989048 19543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.2@KRBTEST.COM'} at 127.18.134.2:57481
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
I20260619 02:50:17.005827 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:0
--local_ip_for_outbound_sockets=127.18.134.3
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:45621
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:46287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:17.319485 19871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:17.319777 19871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:17.319911 19871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:17.327145 19871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:17.327266 19871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:17.327415 19871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:50:17.337090 19871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:46287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:45621
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:17.338379 19871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:17.339771 19871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:17.352141 19877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:17.352161 19876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:17.352705 19879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:17.352953 19871 server_base.cc:1034] running on GCE node
I20260619 02:50:17.353502 19871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:17.354472 19871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:17.355671 19871 hybrid_clock.cc:648] HybridClock initialized: now 1781837417355623 us; error 58 us; skew 500 ppm
I20260619 02:50:17.355952 19871 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:17 dist-test-slave-9mr0 krb5kdc[19498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837417, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:17.358939 19871 init.cc:377] Logged in from keytab as kudu/127.18.134.3@KRBTEST.COM (short username kudu)
I20260619 02:50:17.360325 19871 webserver.cc:460] Webserver started at http://127.18.134.3:35441/ using document root <none> and password file <none>
I20260619 02:50:17.360975 19871 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:17.361079 19871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:17.361369 19871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:17.364319 19871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-2/data/instance:
uuid: "373fa6e747aa435c87282e3377aa52cd"
format_stamp: "Formatted at 2026-06-19 02:50:17 on dist-test-slave-9mr0"
server_key: "dd3f56bbad8f95d52d5ed01d108a6d33"
server_key_iv: "460e203873ca7288b6180807d8d9c400"
server_key_version: "encryptionkey@0"
I20260619 02:50:17.365039 19871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance:
uuid: "373fa6e747aa435c87282e3377aa52cd"
format_stamp: "Formatted at 2026-06-19 02:50:17 on dist-test-slave-9mr0"
server_key: "dd3f56bbad8f95d52d5ed01d108a6d33"
server_key_iv: "460e203873ca7288b6180807d8d9c400"
server_key_version: "encryptionkey@0"
I20260619 02:50:17.370383 19871 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 02:50:17.373749 19886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:17.375168 19871 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:50:17.375314 19871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "373fa6e747aa435c87282e3377aa52cd"
format_stamp: "Formatted at 2026-06-19 02:50:17 on dist-test-slave-9mr0"
server_key: "dd3f56bbad8f95d52d5ed01d108a6d33"
server_key_iv: "460e203873ca7288b6180807d8d9c400"
server_key_version: "encryptionkey@0"
I20260619 02:50:17.375499 19871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:17.393548 19871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:17.397466 19871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:17.397737 19871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:17.398728 19871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:50:17.400040 19871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:17.400110 19871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:17.400223 19871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:17.400270 19871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:17.428535 19871 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:38265
I20260619 02:50:17.428577 19998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:38265 every 8 connection(s)
I20260619 02:50:17.429811 19871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:50:17.431201 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 19871
I20260619 02:50:17.431396 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance
I20260619 02:50:17.431886 18968 external_mini_cluster.cc:1428] Setting key f7157c9187a5bfff0774fa373aa04719
Jun 19 02:50:17 dist-test-slave-9mr0 krb5kdc[19498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837417, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:17.448753 19734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:17.432509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:34927 (local address 127.18.134.62:45621)
0619 02:50:17.432664 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:17.432670 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:17.434140 (+ 1470us) server_negotiation.cc:407] Connection header received
0619 02:50:17.436048 (+ 1908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:17.436054 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:17.436132 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:17.436209 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:17.437654 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.438352 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.439891 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.440278 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.440382 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:17.443617 (+ 3235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:17.443622 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:17.443625 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:17.443656 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:17.446044 (+ 2388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:17.446833 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:17.446838 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:17.446841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:17.446978 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:17.447423 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:17.447427 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:17.447429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:17.447687 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:17.447790 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:17.448362 (+ 572us) server_negotiation.cc:299] Negotiation successful
0619 02:50:17.448520 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 02:50:17.449684 20001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:17.433190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:45621 (local address 127.18.134.3:34927)
0619 02:50:17.433998 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:17.434043 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:17.435806 (+ 1763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:17.436463 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:17.436471 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:17.436961 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:17.437502 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:17.437519 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.438491 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.438497 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:17.439748 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:17.439760 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.440486 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.440493 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:17.440720 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:17.441300 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:17.441348 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:17.443443 (+ 2095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:17.446178 (+ 2735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:17.446186 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:17.446202 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:17.446704 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:17.447094 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:17.447097 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:17.447099 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:17.447309 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:17.447778 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:17.447783 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:17.448080 (+ 297us) client_negotiation.cc:769] Sending connection context
0619 02:50:17.448293 (+ 213us) client_negotiation.cc:240] Negotiation successful
0619 02:50:17.448660 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":306,"threads_started":1}
I20260619 02:50:17.450922 19999 heartbeater.cc:347] Connected to a master server at 127.18.134.62:45621
I20260619 02:50:17.451176 19999 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:17.451850 19999 heartbeater.cc:511] Master 127.18.134.62:45621 requested a full tablet report, sending...
I20260619 02:50:17.453117 19543 ts_manager.cc:194] Registered new tserver with Master: 373fa6e747aa435c87282e3377aa52cd (127.18.134.3:38265)
I20260619 02:50:17.453853 19543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.3@KRBTEST.COM'} at 127.18.134.3:34927
I20260619 02:50:17.459153 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:50:17.476435 19734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:17.465188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52666 (local address 127.18.134.62:45621)
0619 02:50:17.465321 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:17.465325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:17.465738 (+ 413us) server_negotiation.cc:407] Connection header received
0619 02:50:17.465900 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:17.465904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:17.465959 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:17.466034 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:17.467423 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.468111 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.469324 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.469703 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.469805 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:17.471922 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:17.471927 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:17.471930 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:17.471962 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:17.474068 (+ 2106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:17.474654 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:17.474660 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:17.474663 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:17.474783 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:17.475119 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:17.475125 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:17.475128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:17.475428 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:17.475640 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:17.475973 (+ 333us) server_negotiation.cc:299] Negotiation successful
0619 02:50:17.476167 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 02:50:17.482077 19543 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52666:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:50:17.491016 19543 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:50:17.516556 19732 heartbeater.cc:503] Master 127.18.134.62:45621 was elected leader, sending a full tablet report...
I20260619 02:50:17.528746 20015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:17.503838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56196 (local address 127.18.134.2:39057)
0619 02:50:17.520341 (+ 16503us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:17.520347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:17.520371 (+ 24us) server_negotiation.cc:407] Connection header received
0619 02:50:17.520474 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:17.520482 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:17.520763 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:17.521925 (+ 1162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:17.522171 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.523605 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.526144 (+ 2539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.527597 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.527720 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:17.527922 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:17.528262 (+ 340us) server_negotiation.cc:299] Negotiation successful
0619 02:50:17.528435 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16370,"thread_start_us":17072,"threads_started":1}
I20260619 02:50:17.528820 20011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:17.502377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:39057 (local address 127.0.0.1:56196)
0619 02:50:17.508138 (+ 5761us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:17.508186 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:17.508355 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:17.521246 (+ 12891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:17.521253 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:17.521284 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:17.521716 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:17.521729 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.523736 (+ 2007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.523742 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:17.526008 (+ 2266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:17.526019 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.527800 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.527807 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:17.528018 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:17.528070 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 02:50:17.528203 (+ 133us) client_negotiation.cc:240] Negotiation successful
0619 02:50:17.528386 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5499,"mutex_wait_us":47,"thread_start_us":6105,"threads_started":1}
I20260619 02:50:17.529026 20014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:17.502938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37226 (local address 127.18.134.1:45389)
0619 02:50:17.513606 (+ 10668us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:17.513617 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:17.520481 (+ 6864us) server_negotiation.cc:407] Connection header received
0619 02:50:17.521628 (+ 1147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:17.521635 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:17.521889 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:17.522077 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:17.522670 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.524066 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.526316 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.527608 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.527725 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:17.527951 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:17.528598 (+ 647us) server_negotiation.cc:299] Negotiation successful
0619 02:50:17.528745 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10475,"thread_start_us":9018,"threads_started":1}
I20260619 02:50:17.529551 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:17.502577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:45389 (local address 127.0.0.1:37226)
0619 02:50:17.520355 (+ 17778us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:17.520373 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:17.521140 (+ 767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:17.522068 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:17.522073 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:17.522105 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:17.522477 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:17.522489 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.524178 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.524183 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:17.526148 (+ 1965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:17.526160 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.528075 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.528079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:17.528198 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:17.528227 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 02:50:17.528303 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 02:50:17.528430 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17660,"mutex_wait_us":23,"spinlock_wait_cycles":896,"thread_start_us":19146,"threads_started":1}
I20260619 02:50:17.530539 20012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:17.502916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:38265 (local address 127.0.0.1:46574)
0619 02:50:17.521989 (+ 19073us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:17.522005 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:17.522122 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:17.522802 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:17.522806 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:17.522848 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:17.523135 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:17.523144 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.524870 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.524875 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:17.527895 (+ 3020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:17.527908 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.529808 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.529815 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:17.529939 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:17.529959 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 02:50:17.530034 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 02:50:17.530147 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18944,"spinlock_wait_cycles":9472,"thread_start_us":17203,"threads_started":1}
I20260619 02:50:17.530543 20016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:17.503420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46574 (local address 127.18.134.3:38265)
0619 02:50:17.514761 (+ 11341us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:17.514770 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:17.522256 (+ 7486us) server_negotiation.cc:407] Connection header received
0619 02:50:17.522370 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:17.522377 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:17.522607 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:17.522804 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:17.523302 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.524738 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.528020 (+ 3282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.529306 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.529480 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:17.529678 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:17.530091 (+ 413us) server_negotiation.cc:299] Negotiation successful
0619 02:50:17.530241 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11155,"thread_start_us":10412,"threads_started":1}
I20260619 02:50:17.533095 19667 tablet_service.cc:1467] Processing CreateTablet for tablet 4ee35c9cf42544559431c97b17e4904c (DEFAULT_TABLE table=test-table [id=c8d547004a5a4f878d98e8de9997538a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:50:17.533113 19801 tablet_service.cc:1467] Processing CreateTablet for tablet 4ee35c9cf42544559431c97b17e4904c (DEFAULT_TABLE table=test-table [id=c8d547004a5a4f878d98e8de9997538a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:50:17.534269 19667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ee35c9cf42544559431c97b17e4904c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:17.533846 19934 tablet_service.cc:1467] Processing CreateTablet for tablet 4ee35c9cf42544559431c97b17e4904c (DEFAULT_TABLE table=test-table [id=c8d547004a5a4f878d98e8de9997538a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:50:17.534402 19801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ee35c9cf42544559431c97b17e4904c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:17.535036 19934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ee35c9cf42544559431c97b17e4904c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:17.544970 20017 tablet_bootstrap.cc:492] T 4ee35c9cf42544559431c97b17e4904c P d02b24949e184a769e2616739436414b: Bootstrap starting.
I20260619 02:50:17.548087 20018 tablet_bootstrap.cc:492] T 4ee35c9cf42544559431c97b17e4904c P a65a64e51b474735bc40e88329c1a469: Bootstrap starting.
I20260619 02:50:17.548936 20017 tablet_bootstrap.cc:654] T 4ee35c9cf42544559431c97b17e4904c P d02b24949e184a769e2616739436414b: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:17.550212 20017 log.cc:824] T 4ee35c9cf42544559431c97b17e4904c P d02b24949e184a769e2616739436414b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:17.551548 20018 tablet_bootstrap.cc:654] T 4ee35c9cf42544559431c97b17e4904c P a65a64e51b474735bc40e88329c1a469: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:17.553395 20018 log.cc:824] T 4ee35c9cf42544559431c97b17e4904c P a65a64e51b474735bc40e88329c1a469: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:17.554423 20019 tablet_bootstrap.cc:492] T 4ee35c9cf42544559431c97b17e4904c P 373fa6e747aa435c87282e3377aa52cd: Bootstrap starting.
I20260619 02:50:17.556541 20018 tablet_bootstrap.cc:492] T 4ee35c9cf42544559431c97b17e4904c P a65a64e51b474735bc40e88329c1a469: No bootstrap required, opened a new log
I20260619 02:50:17.556816 20018 ts_tablet_manager.cc:1397] T 4ee35c9cf42544559431c97b17e4904c P a65a64e51b474735bc40e88329c1a469: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260619 02:50:17.556852 20017 tablet_bootstrap.cc:492] T 4ee35c9cf42544559431c97b17e4904c P d02b24949e184a769e2616739436414b: No bootstrap required, opened a new log
I20260619 02:50:17.557065 20017 ts_tablet_manager.cc:1397] T 4ee35c9cf42544559431c97b17e4904c P d02b24949e184a769e2616739436414b: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260619 02:50:17.558461 20019 tablet_bootstrap.cc:654] T 4ee35c9cf42544559431c97b17e4904c P 373fa6e747aa435c87282e3377aa52cd: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:17.560154 20019 log.cc:824] T 4ee35c9cf42544559431c97b17e4904c P 373fa6e747aa435c87282e3377aa52cd: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:17.563740 20019 tablet_bootstrap.cc:492] T 4ee35c9cf42544559431c97b17e4904c P 373fa6e747aa435c87282e3377aa52cd: No bootstrap required, opened a new log
I20260619 02:50:17.564005 20019 ts_tablet_manager.cc:1397] T 4ee35c9cf42544559431c97b17e4904c P 373fa6e747aa435c87282e3377aa52cd: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260619 02:50:17.565105 20017 raft_consensus.cc:348] T 4ee35c9cf42544559431c97b17e4904c P d02b24949e184a769e2616739436414b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65a64e51b474735bc40e88329c1a469" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 39057 } } peers { permanent_uuid: "d02b24949e184a769e2616739436414b" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45389 } } peers { permanent_uuid: "373fa6e747aa435c87282e3377aa52cd" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 38265 } }
I20260619 02:50:17.565435 20017 raft_consensus.cc:374] T 4ee35c9cf42544559431c97b17e4904c P d02b24949e184a769e2616739436414b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:17.565539 20017 raft_consensus.cc:729] T 4ee35c9cf42544559431c97b17e4904c P d02b24949e184a769e2616739436414b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02b24949e184a769e2616739436414b, State: Initialized, Role: FOLLOWER
I20260619 02:50:17.566088 20017 consensus_queue.cc:260] T 4ee35c9cf42544559431c97b17e4904c P d02b24949e184a769e2616739436414b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65a64e51b474735bc40e88329c1a469" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 39057 } } peers { permanent_uuid: "d02b24949e184a769e2616739436414b" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45389 } } peers { permanent_uuid: "373fa6e747aa435c87282e3377aa52cd" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 38265 } }
I20260619 02:50:17.566725 20018 raft_consensus.cc:348] T 4ee35c9cf42544559431c97b17e4904c P a65a64e51b474735bc40e88329c1a469 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65a64e51b474735bc40e88329c1a469" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 39057 } } peers { permanent_uuid: "d02b24949e184a769e2616739436414b" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45389 } } peers { permanent_uuid: "373fa6e747aa435c87282e3377aa52cd" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 38265 } }
I20260619 02:50:17.567327 20018 raft_consensus.cc:374] T 4ee35c9cf42544559431c97b17e4904c P a65a64e51b474735bc40e88329c1a469 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:17.567445 20018 raft_consensus.cc:729] T 4ee35c9cf42544559431c97b17e4904c P a65a64e51b474735bc40e88329c1a469 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a65a64e51b474735bc40e88329c1a469, State: Initialized, Role: FOLLOWER
I20260619 02:50:17.567684 20017 ts_tablet_manager.cc:1428] T 4ee35c9cf42544559431c97b17e4904c P d02b24949e184a769e2616739436414b: Time spent starting tablet: real 0.011s user 0.006s sys 0.005s
I20260619 02:50:17.568051 20018 consensus_queue.cc:260] T 4ee35c9cf42544559431c97b17e4904c P a65a64e51b474735bc40e88329c1a469 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65a64e51b474735bc40e88329c1a469" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 39057 } } peers { permanent_uuid: "d02b24949e184a769e2616739436414b" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45389 } } peers { permanent_uuid: "373fa6e747aa435c87282e3377aa52cd" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 38265 } }
I20260619 02:50:17.569072 19866 heartbeater.cc:503] Master 127.18.134.62:45621 was elected leader, sending a full tablet report...
I20260619 02:50:17.569641 20018 ts_tablet_manager.cc:1428] T 4ee35c9cf42544559431c97b17e4904c P a65a64e51b474735bc40e88329c1a469: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260619 02:50:17.573724 20019 raft_consensus.cc:348] T 4ee35c9cf42544559431c97b17e4904c P 373fa6e747aa435c87282e3377aa52cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65a64e51b474735bc40e88329c1a469" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 39057 } } peers { permanent_uuid: "d02b24949e184a769e2616739436414b" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45389 } } peers { permanent_uuid: "373fa6e747aa435c87282e3377aa52cd" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 38265 } }
I20260619 02:50:17.574115 20019 raft_consensus.cc:374] T 4ee35c9cf42544559431c97b17e4904c P 373fa6e747aa435c87282e3377aa52cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:17.574208 20019 raft_consensus.cc:729] T 4ee35c9cf42544559431c97b17e4904c P 373fa6e747aa435c87282e3377aa52cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373fa6e747aa435c87282e3377aa52cd, State: Initialized, Role: FOLLOWER
I20260619 02:50:17.574808 20019 consensus_queue.cc:260] T 4ee35c9cf42544559431c97b17e4904c P 373fa6e747aa435c87282e3377aa52cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65a64e51b474735bc40e88329c1a469" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 39057 } } peers { permanent_uuid: "d02b24949e184a769e2616739436414b" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45389 } } peers { permanent_uuid: "373fa6e747aa435c87282e3377aa52cd" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 38265 } }
I20260619 02:50:17.575680 19999 heartbeater.cc:503] Master 127.18.134.62:45621 was elected leader, sending a full tablet report...
I20260619 02:50:17.576076 20019 ts_tablet_manager.cc:1428] T 4ee35c9cf42544559431c97b17e4904c P 373fa6e747aa435c87282e3377aa52cd: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
W20260619 02:50:17.681537 20000 tablet.cc:2369] T 4ee35c9cf42544559431c97b17e4904c P 373fa6e747aa435c87282e3377aa52cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:50:17.713289 19867 tablet.cc:2369] T 4ee35c9cf42544559431c97b17e4904c P a65a64e51b474735bc40e88329c1a469: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:50:17.717099 20023 raft_consensus.cc:482] T 4ee35c9cf42544559431c97b17e4904c P d02b24949e184a769e2616739436414b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:50:17.717356 20023 raft_consensus.cc:504] T 4ee35c9cf42544559431c97b17e4904c P d02b24949e184a769e2616739436414b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65a64e51b474735bc40e88329c1a469" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 39057 } } peers { permanent_uuid: "d02b24949e184a769e2616739436414b" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45389 } } peers { permanent_uuid: "373fa6e747aa435c87282e3377aa52cd" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 38265 } }
I20260619 02:50:17.719024 20023 leader_election.cc:288] T 4ee35c9cf42544559431c97b17e4904c P d02b24949e184a769e2616739436414b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a65a64e51b474735bc40e88329c1a469 (127.18.134.2:39057), 373fa6e747aa435c87282e3377aa52cd (127.18.134.3:38265)
I20260619 02:50:17.725502 20015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:17.719412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:45731 (local address 127.18.134.2:39057)
0619 02:50:17.719711 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:17.719716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:17.720105 (+ 389us) server_negotiation.cc:407] Connection header received
0619 02:50:17.720392 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:17.720398 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:17.720466 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:17.720530 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:17.721086 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.721775 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.723781 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.724520 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.724632 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:17.724699 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:17.725146 (+ 447us) server_negotiation.cc:299] Negotiation successful
0619 02:50:17.725265 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170}
I20260619 02:50:17.725539 20016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:17.719504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:54223 (local address 127.18.134.3:38265)
0619 02:50:17.719664 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:17.719671 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:17.720292 (+ 621us) server_negotiation.cc:407] Connection header received
0619 02:50:17.720461 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:17.720465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:17.720541 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:17.720624 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:17.721224 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.722064 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.723796 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.724575 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.724748 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:17.724827 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:17.725157 (+ 330us) server_negotiation.cc:299] Negotiation successful
0619 02:50:17.725281 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 02:50:17.725626 20027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:17.719457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:38265 (local address 127.18.134.1:54223)
0619 02:50:17.720190 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:17.720204 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:17.720319 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:17.720651 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:17.720656 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:17.720681 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:17.721035 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:17.721046 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.722209 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.722212 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:17.723636 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:17.723647 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.724790 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.724795 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:17.724932 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:17.725017 (+ 85us) client_negotiation.cc:769] Sending connection context
0619 02:50:17.725094 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 02:50:17.725216 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"mutex_wait_us":55,"thread_start_us":414,"threads_started":1}
I20260619 02:50:17.725692 20026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:17.719250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:39057 (local address 127.18.134.1:45731)
0619 02:50:17.719970 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:17.719990 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:17.720197 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:17.720574 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:17.720578 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:17.720611 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:17.720950 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:17.720959 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.721923 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.721930 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:17.723623 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:17.723636 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.724756 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.724763 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:17.724972 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:17.725015 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 02:50:17.725099 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 02:50:17.725232 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"mutex_wait_us":32,"thread_start_us":421,"threads_started":1}
I20260619 02:50:17.726686 19821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ee35c9cf42544559431c97b17e4904c" candidate_uuid: "d02b24949e184a769e2616739436414b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a65a64e51b474735bc40e88329c1a469" is_pre_election: true
I20260619 02:50:17.726567 19954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ee35c9cf42544559431c97b17e4904c" candidate_uuid: "d02b24949e184a769e2616739436414b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373fa6e747aa435c87282e3377aa52cd" is_pre_election: true
I20260619 02:50:17.727348 19821 raft_consensus.cc:2454] T 4ee35c9cf42544559431c97b17e4904c P a65a64e51b474735bc40e88329c1a469 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02b24949e184a769e2616739436414b in term 0.
I20260619 02:50:17.727677 19954 raft_consensus.cc:2454] T 4ee35c9cf42544559431c97b17e4904c P 373fa6e747aa435c87282e3377aa52cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d02b24949e184a769e2616739436414b in term 0.
I20260619 02:50:17.728018 19623 leader_election.cc:302] T 4ee35c9cf42544559431c97b17e4904c P d02b24949e184a769e2616739436414b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a65a64e51b474735bc40e88329c1a469, d02b24949e184a769e2616739436414b; no voters:
I20260619 02:50:17.728369 20023 raft_consensus.cc:2781] T 4ee35c9cf42544559431c97b17e4904c P d02b24949e184a769e2616739436414b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:50:17.728487 20023 raft_consensus.cc:482] T 4ee35c9cf42544559431c97b17e4904c P d02b24949e184a769e2616739436414b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:50:17.728576 20023 raft_consensus.cc:3037] T 4ee35c9cf42544559431c97b17e4904c P d02b24949e184a769e2616739436414b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:17.730767 20023 raft_consensus.cc:504] T 4ee35c9cf42544559431c97b17e4904c P d02b24949e184a769e2616739436414b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65a64e51b474735bc40e88329c1a469" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 39057 } } peers { permanent_uuid: "d02b24949e184a769e2616739436414b" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45389 } } peers { permanent_uuid: "373fa6e747aa435c87282e3377aa52cd" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 38265 } }
I20260619 02:50:17.731298 20023 leader_election.cc:288] T 4ee35c9cf42544559431c97b17e4904c P d02b24949e184a769e2616739436414b [CANDIDATE]: Term 1 election: Requested vote from peers a65a64e51b474735bc40e88329c1a469 (127.18.134.2:39057), 373fa6e747aa435c87282e3377aa52cd (127.18.134.3:38265)
I20260619 02:50:17.731681 19821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ee35c9cf42544559431c97b17e4904c" candidate_uuid: "d02b24949e184a769e2616739436414b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a65a64e51b474735bc40e88329c1a469"
I20260619 02:50:17.731739 19954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ee35c9cf42544559431c97b17e4904c" candidate_uuid: "d02b24949e184a769e2616739436414b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373fa6e747aa435c87282e3377aa52cd"
I20260619 02:50:17.731848 19821 raft_consensus.cc:3037] T 4ee35c9cf42544559431c97b17e4904c P a65a64e51b474735bc40e88329c1a469 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:17.731925 19954 raft_consensus.cc:3037] T 4ee35c9cf42544559431c97b17e4904c P 373fa6e747aa435c87282e3377aa52cd [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:17.733873 19821 raft_consensus.cc:2454] T 4ee35c9cf42544559431c97b17e4904c P a65a64e51b474735bc40e88329c1a469 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02b24949e184a769e2616739436414b in term 1.
I20260619 02:50:17.733958 19954 raft_consensus.cc:2454] T 4ee35c9cf42544559431c97b17e4904c P 373fa6e747aa435c87282e3377aa52cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d02b24949e184a769e2616739436414b in term 1.
I20260619 02:50:17.734195 19623 leader_election.cc:302] T 4ee35c9cf42544559431c97b17e4904c P d02b24949e184a769e2616739436414b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a65a64e51b474735bc40e88329c1a469, d02b24949e184a769e2616739436414b; no voters:
I20260619 02:50:17.734473 20023 raft_consensus.cc:2781] T 4ee35c9cf42544559431c97b17e4904c P d02b24949e184a769e2616739436414b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:50:17.734907 20023 raft_consensus.cc:686] T 4ee35c9cf42544559431c97b17e4904c P d02b24949e184a769e2616739436414b [term 1 LEADER]: Becoming Leader. State: Replica: d02b24949e184a769e2616739436414b, State: Running, Role: LEADER
I20260619 02:50:17.735409 20023 consensus_queue.cc:237] T 4ee35c9cf42544559431c97b17e4904c P d02b24949e184a769e2616739436414b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65a64e51b474735bc40e88329c1a469" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 39057 } } peers { permanent_uuid: "d02b24949e184a769e2616739436414b" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45389 } } peers { permanent_uuid: "373fa6e747aa435c87282e3377aa52cd" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 38265 } }
W20260619 02:50:17.738807 19733 tablet.cc:2369] T 4ee35c9cf42544559431c97b17e4904c P d02b24949e184a769e2616739436414b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:50:17.739526 19543 catalog_manager.cc:5495] T 4ee35c9cf42544559431c97b17e4904c P d02b24949e184a769e2616739436414b reported cstate change: term changed from 0 to 1, leader changed from <none> to d02b24949e184a769e2616739436414b (127.18.134.1). New cstate: current_term: 1 leader_uuid: "d02b24949e184a769e2616739436414b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a65a64e51b474735bc40e88329c1a469" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 39057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d02b24949e184a769e2616739436414b" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "373fa6e747aa435c87282e3377aa52cd" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 38265 } health_report { overall_health: UNKNOWN } } }
Jun 19 02:50:17 dist-test-slave-9mr0 krb5kdc[19498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.1@KRBTEST.COM
I20260619 02:50:17.762113 20014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:17.748837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37232 (local address 127.18.134.1:45389)
0619 02:50:17.749004 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:17.749010 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:17.749393 (+ 383us) server_negotiation.cc:407] Connection header received
0619 02:50:17.749568 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:17.749573 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:17.749629 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:17.749704 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:17.751089 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.751765 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.752938 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.753315 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.753419 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:17.756960 (+ 3541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:17.756973 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:17.756987 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:17.757020 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:17.759586 (+ 2566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:17.760187 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:17.760194 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:17.760196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:17.760354 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:17.760708 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:17.760712 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:17.760714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:17.761045 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:17.761299 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:17.761586 (+ 287us) server_negotiation.cc:299] Negotiation successful
0619 02:50:17.761791 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Jun 19 02:50:17 dist-test-slave-9mr0 krb5kdc[19498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837417, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 02:50:17 dist-test-slave-9mr0 krb5kdc[19498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837417, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.134.1@KRBTEST.COM
I20260619 02:50:17.803244 20014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:17.790491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37238 (local address 127.18.134.1:45389)
0619 02:50:17.790776 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:17.790783 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:17.791071 (+ 288us) server_negotiation.cc:407] Connection header received
0619 02:50:17.791208 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:17.791211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:17.791277 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:17.791363 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:17.792598 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.793311 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.794499 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.794955 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.795046 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:17.798425 (+ 3379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:17.798429 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:17.798433 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:17.798467 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:17.800874 (+ 2407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:17.801443 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:17.801447 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:17.801449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:17.801564 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:17.801947 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:17.801951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:17.801953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:17.802210 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:17.802477 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:17.802718 (+ 241us) server_negotiation.cc:299] Negotiation successful
0619 02:50:17.802925 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
Jun 19 02:50:17 dist-test-slave-9mr0 krb5kdc[19498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 02:50:17 dist-test-slave-9mr0 krb5kdc[19498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.1@KRBTEST.COM
I20260619 02:50:17.841595 20014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:17.828635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37252 (local address 127.18.134.1:45389)
0619 02:50:17.828795 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:17.828801 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:17.829193 (+ 392us) server_negotiation.cc:407] Connection header received
0619 02:50:17.829355 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:17.829359 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:17.829435 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:17.829517 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:17.831255 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.831988 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.833133 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.833520 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.833621 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:17.837029 (+ 3408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:17.837034 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:17.837037 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:17.837069 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:17.839317 (+ 2248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:17.839886 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:17.839890 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:17.839893 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:17.840011 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:17.840330 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:17.840336 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:17.840339 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:17.840597 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:17.840819 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:17.841087 (+ 268us) server_negotiation.cc:299] Negotiation successful
0619 02:50:17.841279 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260619 02:50:17.843225 19647 tablet_service.cc:2908] Rejecting scan request for tablet 4ee35c9cf42544559431c97b17e4904c: Uninitialized: clean time has not yet been initialized
I20260619 02:50:17.845988 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 19604
I20260619 02:50:17.857240 20014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:17.846798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37254 (local address 127.18.134.1:45389)
0619 02:50:17.846949 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:17.846953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:17.847271 (+ 318us) server_negotiation.cc:407] Connection header received
0619 02:50:17.847394 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:17.847397 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:17.847450 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:17.847522 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:17.848536 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.849198 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.850244 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.850618 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.850741 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:17.852597 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:17.852601 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:17.852605 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:17.852631 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:17.854925 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:17.855455 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:17.855458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:17.855461 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:17.855568 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:17.855816 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:17.855818 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:17.855820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:17.856055 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:17.856218 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:17.856410 (+ 192us) server_negotiation.cc:299] Negotiation successful
0619 02:50:17.856999 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 02:50:17.857853 19727 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:17.946826 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 19604
I20260619 02:50:17.960808 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 19738
Jun 19 02:50:17 dist-test-slave-9mr0 krb5kdc[19498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.2@KRBTEST.COM
I20260619 02:50:17.975257 20015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:17.961556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56208 (local address 127.18.134.2:39057)
0619 02:50:17.961740 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:17.961745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:17.961783 (+ 38us) server_negotiation.cc:407] Connection header received
0619 02:50:17.961955 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:17.961961 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:17.962034 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:17.962138 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:17.963325 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.964147 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.965362 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:17.965773 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:17.965876 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:17.969616 (+ 3740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:17.969627 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:17.969640 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:17.969672 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:17.972504 (+ 2832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:17.973143 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:17.973151 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:17.973153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:17.973282 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:17.973608 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:17.973613 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:17.973615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:17.973966 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:17.974248 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:17.974511 (+ 263us) server_negotiation.cc:299] Negotiation successful
0619 02:50:17.974973 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 02:50:17.975878 19861 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:18.060310 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 19738
I20260619 02:50:18.072556 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 19871
Jun 19 02:50:18 dist-test-slave-9mr0 krb5kdc[19498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.3@KRBTEST.COM
I20260619 02:50:18.086107 20016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:18.073279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46578 (local address 127.18.134.3:38265)
0619 02:50:18.073435 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:18.073439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:18.073459 (+ 20us) server_negotiation.cc:407] Connection header received
0619 02:50:18.073603 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:18.073607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:18.073678 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:18.073758 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:18.074911 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:18.075635 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:18.077115 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:18.077480 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:18.077572 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:18.081432 (+ 3860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:18.081444 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:18.081460 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:18.081495 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:18.083896 (+ 2401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:18.084477 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:18.084484 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:18.084486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:18.084617 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:18.084881 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:18.084884 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:18.084886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:18.085189 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:18.085391 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:18.085616 (+ 225us) server_negotiation.cc:299] Negotiation successful
0619 02:50:18.085775 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 02:50:18.086869 19994 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:18.171759 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 19871
I20260619 02:50:18.183899 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 19511
I20260619 02:50:18.185039 19573 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:18.289021 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 19511
2026-06-19T02:50:18Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (3914 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:50:18 dist-test-slave-9mr0 krb5kdc[20047](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:50:18 dist-test-slave-9mr0 krb5kdc[20047](info): set up 2 sockets
Jun 19 02:50:18 dist-test-slave-9mr0 krb5kdc[20047](info): commencing operation
krb5kdc: starting...
W20260619 02:50:19.282440 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.957s user 0.000s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:50:19 dist-test-slave-9mr0 krb5kdc[20047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:50:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:50:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:50:19.450130 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:39567
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:44663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:39567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:19.759883 20060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:19.760200 20060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:19.760335 20060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:19.767575 20060 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:50:19.767696 20060 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 02:50:19.767762 20060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:19.767856 20060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:19.767920 20060 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:50:19.768003 20060 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:50:19.777868 20060 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:44663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:39567
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:39567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:19.779150 20060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:19.780441 20060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:19.791574 20065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:19.791602 20068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:19.791630 20066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:19.792732 20060 server_base.cc:1034] running on GCE node
I20260619 02:50:19.793493 20060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:19.794864 20060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:19.796072 20060 hybrid_clock.cc:648] HybridClock initialized: now 1781837419796012 us; error 42 us; skew 500 ppm
I20260619 02:50:19.796377 20060 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:19 dist-test-slave-9mr0 krb5kdc[20047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837419, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:19.799256 20060 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:50:19.800671 20060 webserver.cc:460] Webserver started at http://127.18.134.62:44195/ using document root <none> and password file <none>
I20260619 02:50:19.801316 20060 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:19.801395 20060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:19.801654 20060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:19.804495 20060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "a6c62c7df18c4c549706d19cd35f702a"
format_stamp: "Formatted at 2026-06-19 02:50:19 on dist-test-slave-9mr0"
server_key: "33508ae3c9e6e502418fcadeb2da665d"
server_key_iv: "8f5ae05a7812207f85b8c46099894fe3"
server_key_version: "encryptionkey@0"
I20260619 02:50:19.805182 20060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "a6c62c7df18c4c549706d19cd35f702a"
format_stamp: "Formatted at 2026-06-19 02:50:19 on dist-test-slave-9mr0"
server_key: "33508ae3c9e6e502418fcadeb2da665d"
server_key_iv: "8f5ae05a7812207f85b8c46099894fe3"
server_key_version: "encryptionkey@0"
I20260619 02:50:19.810199 20060 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260619 02:50:19.813674 20075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:19.815109 20060 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:50:19.815254 20060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "a6c62c7df18c4c549706d19cd35f702a"
format_stamp: "Formatted at 2026-06-19 02:50:19 on dist-test-slave-9mr0"
server_key: "33508ae3c9e6e502418fcadeb2da665d"
server_key_iv: "8f5ae05a7812207f85b8c46099894fe3"
server_key_version: "encryptionkey@0"
I20260619 02:50:19.815452 20060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:19.841369 20060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:19.845619 20060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:19.845878 20060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:19.865590 20060 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:39567
I20260619 02:50:19.865608 20136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:39567 every 8 connection(s)
I20260619 02:50:19.866920 20060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:50:19.870563 20137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:19.875236 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 20060
I20260619 02:50:19.875438 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:50:19.876053 18968 external_mini_cluster.cc:1428] Setting key 197aa0c9e3cccf286ba5e0f498f04c77
I20260619 02:50:19.881371 20137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6c62c7df18c4c549706d19cd35f702a: Bootstrap starting.
I20260619 02:50:19.885146 20137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6c62c7df18c4c549706d19cd35f702a: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:19.886420 20137 log.cc:824] T 00000000000000000000000000000000 P a6c62c7df18c4c549706d19cd35f702a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:19.889909 20137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6c62c7df18c4c549706d19cd35f702a: No bootstrap required, opened a new log
Jun 19 02:50:19 dist-test-slave-9mr0 krb5kdc[20047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:19.899123 20137 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6c62c7df18c4c549706d19cd35f702a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6c62c7df18c4c549706d19cd35f702a" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39567 } }
I20260619 02:50:19.899492 20137 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6c62c7df18c4c549706d19cd35f702a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:19.899588 20137 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6c62c7df18c4c549706d19cd35f702a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6c62c7df18c4c549706d19cd35f702a, State: Initialized, Role: FOLLOWER
I20260619 02:50:19.899717 20140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:19.878100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39450 (local address 127.18.134.62:39567)
0619 02:50:19.878919 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:19.878934 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:19.878974 (+ 40us) server_negotiation.cc:407] Connection header received
0619 02:50:19.880647 (+ 1673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:19.880670 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:19.881093 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:19.881440 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:19.883161 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:19.884750 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:19.886501 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:19.887537 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:19.887722 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:19.892404 (+ 4682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:19.892412 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:19.892428 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:19.892473 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:19.895531 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:19.896339 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:19.896346 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:19.896365 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:19.896528 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:19.896898 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:19.896901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:19.896903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:19.897465 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:19.897670 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:19.898121 (+ 451us) server_negotiation.cc:299] Negotiation successful
0619 02:50:19.898514 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":342,"threads_started":1}
I20260619 02:50:19.900202 20137 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6c62c7df18c4c549706d19cd35f702a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6c62c7df18c4c549706d19cd35f702a" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39567 } }
I20260619 02:50:19.900440 20137 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6c62c7df18c4c549706d19cd35f702a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:50:19.900564 20137 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6c62c7df18c4c549706d19cd35f702a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:50:19.900719 20137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6c62c7df18c4c549706d19cd35f702a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:19.903152 20137 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6c62c7df18c4c549706d19cd35f702a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6c62c7df18c4c549706d19cd35f702a" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39567 } }
I20260619 02:50:19.903628 20137 leader_election.cc:302] T 00000000000000000000000000000000 P a6c62c7df18c4c549706d19cd35f702a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6c62c7df18c4c549706d19cd35f702a; no voters:
I20260619 02:50:19.904266 20137 leader_election.cc:288] T 00000000000000000000000000000000 P a6c62c7df18c4c549706d19cd35f702a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:50:19.904495 20142 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6c62c7df18c4c549706d19cd35f702a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:50:19.905539 20142 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6c62c7df18c4c549706d19cd35f702a [term 1 LEADER]: Becoming Leader. State: Replica: a6c62c7df18c4c549706d19cd35f702a, State: Running, Role: LEADER
I20260619 02:50:19.906020 20142 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6c62c7df18c4c549706d19cd35f702a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6c62c7df18c4c549706d19cd35f702a" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39567 } }
I20260619 02:50:19.906219 20137 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6c62c7df18c4c549706d19cd35f702a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:50:19.908833 20144 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6c62c7df18c4c549706d19cd35f702a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6c62c7df18c4c549706d19cd35f702a. Latest consensus state: current_term: 1 leader_uuid: "a6c62c7df18c4c549706d19cd35f702a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6c62c7df18c4c549706d19cd35f702a" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39567 } } }
I20260619 02:50:19.909217 20144 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6c62c7df18c4c549706d19cd35f702a [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:19.909968 20143 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6c62c7df18c4c549706d19cd35f702a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6c62c7df18c4c549706d19cd35f702a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6c62c7df18c4c549706d19cd35f702a" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39567 } } }
I20260619 02:50:19.910248 20143 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6c62c7df18c4c549706d19cd35f702a [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:19.910905 20151 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:50:19.915859 20151 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:50:19.925227 20151 catalog_manager.cc:1347] Generated new cluster ID: 214641c1fd354ad395bbd296d6307304
I20260619 02:50:19.925410 20151 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:50:19.932240 20151 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:50:19.933522 20151 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:50:19.939527 20151 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6c62c7df18c4c549706d19cd35f702a: Generated new TSK 0
I20260619 02:50:19.940418 20151 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.134.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
I20260619 02:50:19.982175 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:0
--local_ip_for_outbound_sockets=127.18.134.1
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:39567
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:44663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:20.295995 20163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:20.296289 20163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:20.296391 20163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:20.303529 20163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:20.303647 20163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:20.303727 20163 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:50:20.303804 20163 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:50:20.313699 20163 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:44663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.134.62:39567
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:20.315001 20163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:20.316259 20163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:20.329077 20169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:20.329033 20171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:20.329012 20168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:20.330027 20163 server_base.cc:1034] running on GCE node
I20260619 02:50:20.330701 20163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:20.331625 20163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:20.332835 20163 hybrid_clock.cc:648] HybridClock initialized: now 1781837420332777 us; error 37 us; skew 500 ppm
I20260619 02:50:20.333145 20163 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:20 dist-test-slave-9mr0 krb5kdc[20047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837420, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:20.336191 20163 init.cc:377] Logged in from keytab as kudu/127.18.134.1@KRBTEST.COM (short username kudu)
I20260619 02:50:20.337661 20163 webserver.cc:460] Webserver started at http://127.18.134.1:46701/ using document root <none> and password file <none>
I20260619 02:50:20.338359 20163 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:20.338456 20163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:20.338798 20163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:20.341727 20163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-0/data/instance:
uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d"
format_stamp: "Formatted at 2026-06-19 02:50:20 on dist-test-slave-9mr0"
server_key: "d7e2ce36f190a763765b099fe17bbdc1"
server_key_iv: "5e8faff5c48662d18ccd5cd6b0c354d2"
server_key_version: "encryptionkey@0"
I20260619 02:50:20.342540 20163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance:
uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d"
format_stamp: "Formatted at 2026-06-19 02:50:20 on dist-test-slave-9mr0"
server_key: "d7e2ce36f190a763765b099fe17bbdc1"
server_key_iv: "5e8faff5c48662d18ccd5cd6b0c354d2"
server_key_version: "encryptionkey@0"
I20260619 02:50:20.347823 20163 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 02:50:20.351395 20178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:20.352843 20163 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:50:20.353020 20163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d"
format_stamp: "Formatted at 2026-06-19 02:50:20 on dist-test-slave-9mr0"
server_key: "d7e2ce36f190a763765b099fe17bbdc1"
server_key_iv: "5e8faff5c48662d18ccd5cd6b0c354d2"
server_key_version: "encryptionkey@0"
I20260619 02:50:20.353238 20163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:20.379632 20163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:20.383705 20163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:20.383973 20163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:20.386971 20163 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:20.387065 20163 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:20.387230 20163 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:20.387305 20163 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:50:20 dist-test-slave-9mr0 krb5kdc[20047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837420, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:20.411357 20140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:20.387775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:34099 (local address 127.18.134.62:39567)
0619 02:50:20.387942 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:20.387947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:20.389737 (+ 1790us) server_negotiation.cc:407] Connection header received
0619 02:50:20.392084 (+ 2347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:20.392090 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:20.392178 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:20.392267 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:20.394503 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:20.395488 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:20.397792 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:20.398315 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:20.398448 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:20.403072 (+ 4624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:20.403079 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:20.403083 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:20.403124 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:20.406698 (+ 3574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:20.407974 (+ 1276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:20.407981 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:20.407984 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:20.408145 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:20.408756 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:20.408760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:20.408764 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:20.409084 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:20.409209 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:20.410798 (+ 1589us) server_negotiation.cc:299] Negotiation successful
0619 02:50:20.411017 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 02:50:20.412824 20187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:20.388629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39567 (local address 127.18.134.1:34099)
0619 02:50:20.389536 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:20.389586 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:20.391739 (+ 2153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:20.392586 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:20.392602 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:20.393403 (+ 801us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:20.394314 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:20.394338 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:20.395645 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:20.395649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:20.397578 (+ 1929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:20.397597 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:20.398582 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:20.398594 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:20.398923 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:20.399779 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:20.399817 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:20.402834 (+ 3017us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:20.406902 (+ 4068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:20.406916 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:20.406945 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:20.407778 (+ 833us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:20.408281 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:20.408286 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:20.408291 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:20.408631 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:20.409221 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:20.409230 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:20.409810 (+ 580us) client_negotiation.cc:769] Sending connection context
0619 02:50:20.410247 (+ 437us) client_negotiation.cc:240] Negotiation successful
0619 02:50:20.410991 (+ 744us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":386,"threads_started":1}
I20260619 02:50:20.428494 20163 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:36831
I20260619 02:50:20.428570 20297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:36831 every 8 connection(s)
I20260619 02:50:20.429826 20163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:50:20.431948 20298 heartbeater.cc:347] Connected to a master server at 127.18.134.62:39567
I20260619 02:50:20.432044 20273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:20.418815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39567 (local address 127.18.134.1:56463)
0619 02:50:20.419496 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:20.419522 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:20.419688 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:20.420015 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:20.420020 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:20.420494 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:20.420936 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:20.420949 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:20.422150 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:20.422156 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:20.423470 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:20.423484 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:20.424576 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:20.424585 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:20.424765 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:20.425467 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:20.425492 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:20.426491 (+ 999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:20.429477 (+ 2986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:20.429482 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:20.429487 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:20.429861 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:20.430295 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:20.430301 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:20.430304 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:20.430466 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:20.431125 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:20.431130 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:20.431321 (+ 191us) client_negotiation.cc:769] Sending connection context
0619 02:50:20.431442 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 02:50:20.431675 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":346,"threads_started":1}
I20260619 02:50:20.432193 20298 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:20.432750 20140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:20.418682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:56463 (local address 127.18.134.62:39567)
0619 02:50:20.418851 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:20.418857 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:20.419604 (+ 747us) server_negotiation.cc:407] Connection header received
0619 02:50:20.419806 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:20.419810 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:20.419883 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:20.419970 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:20.421085 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:20.422017 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:20.423652 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:20.424215 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:20.424334 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:20.426687 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:20.426693 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:20.426698 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:20.426732 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:20.429356 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:20.430024 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:20.430029 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:20.430033 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:20.430179 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:20.430597 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:20.430602 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:20.430606 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:20.430989 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:20.432286 (+ 1297us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:20.432389 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 02:50:20.432545 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 02:50:20.432854 20298 heartbeater.cc:511] Master 127.18.134.62:39567 requested a full tablet report, sending...
I20260619 02:50:20.434829 20092 ts_manager.cc:194] Registered new tserver with Master: 0c2b46b6e18547e99f0c75cd3eb9e09d (127.18.134.1:36831)
I20260619 02:50:20.436237 20092 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.1@KRBTEST.COM'} at 127.18.134.1:34099
I20260619 02:50:20.438799 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 20163
I20260619 02:50:20.439029 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance
I20260619 02:50:20.439678 18968 external_mini_cluster.cc:1428] Setting key fdc8e41cdbba8d495c7123b5cb5197eb
WARNING: no policy specified for kudu/127.18.134.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
I20260619 02:50:20.473701 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:0
--local_ip_for_outbound_sockets=127.18.134.2
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:39567
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:44663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:20.786206 20302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:20.786504 20302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:20.786602 20302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:20.794021 20302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:20.794147 20302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:20.794230 20302 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:50:20.794309 20302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:50:20.804322 20302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:44663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.134.62:39567
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:20.805557 20302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:20.806983 20302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:20.820043 20310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:20.820145 20307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:20.820106 20308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:20.820429 20302 server_base.cc:1034] running on GCE node
I20260619 02:50:20.821301 20302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:20.822216 20302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:20.823422 20302 hybrid_clock.cc:648] HybridClock initialized: now 1781837420823352 us; error 61 us; skew 500 ppm
I20260619 02:50:20.823710 20302 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:20 dist-test-slave-9mr0 krb5kdc[20047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837420, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:20.826687 20302 init.cc:377] Logged in from keytab as kudu/127.18.134.2@KRBTEST.COM (short username kudu)
I20260619 02:50:20.828060 20302 webserver.cc:460] Webserver started at http://127.18.134.2:46021/ using document root <none> and password file <none>
I20260619 02:50:20.828725 20302 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:20.828820 20302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:20.829118 20302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:20.832101 20302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-1/data/instance:
uuid: "2f694c40a66c412b879c04f7c183cfb5"
format_stamp: "Formatted at 2026-06-19 02:50:20 on dist-test-slave-9mr0"
server_key: "4625741998f13c0b4c4409e65d1c6532"
server_key_iv: "df3fab6c53c199fd0ab1aa7a632849d8"
server_key_version: "encryptionkey@0"
I20260619 02:50:20.832876 20302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance:
uuid: "2f694c40a66c412b879c04f7c183cfb5"
format_stamp: "Formatted at 2026-06-19 02:50:20 on dist-test-slave-9mr0"
server_key: "4625741998f13c0b4c4409e65d1c6532"
server_key_iv: "df3fab6c53c199fd0ab1aa7a632849d8"
server_key_version: "encryptionkey@0"
I20260619 02:50:20.838016 20302 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 02:50:20.841404 20317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:20.842741 20302 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:50:20.842903 20302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "2f694c40a66c412b879c04f7c183cfb5"
format_stamp: "Formatted at 2026-06-19 02:50:20 on dist-test-slave-9mr0"
server_key: "4625741998f13c0b4c4409e65d1c6532"
server_key_iv: "df3fab6c53c199fd0ab1aa7a632849d8"
server_key_version: "encryptionkey@0"
I20260619 02:50:20.843094 20302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:20.859586 20302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:20.863930 20302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:20.864159 20302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:20.867200 20302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:20.867381 20302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260619 02:50:20.867578 20302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:20.867682 20302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:50:20 dist-test-slave-9mr0 krb5kdc[20047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837420, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:20.893455 20140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:20.867403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:44361 (local address 127.18.134.62:39567)
0619 02:50:20.867594 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:20.867600 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:20.869072 (+ 1472us) server_negotiation.cc:407] Connection header received
0619 02:50:20.871256 (+ 2184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:20.871262 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:20.871345 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:20.871437 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:20.873032 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:20.874031 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:20.877061 (+ 3030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:20.877716 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:20.877844 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:20.887173 (+ 9329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:20.887181 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:20.887187 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:20.887233 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:20.890148 (+ 2915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:20.891045 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:20.891051 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:20.891054 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:20.891179 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:20.891657 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:20.891662 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:20.891664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:20.891944 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:20.892070 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:20.892914 (+ 844us) server_negotiation.cc:299] Negotiation successful
0619 02:50:20.893144 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 02:50:20.894953 20326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:20.867999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39567 (local address 127.18.134.2:44361)
0619 02:50:20.868909 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:20.868959 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:20.870984 (+ 2025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:20.871689 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:20.871697 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:20.872225 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:20.872864 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:20.872883 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:20.874175 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:20.874181 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:20.876834 (+ 2653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:20.876865 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:20.877957 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:20.877967 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:20.878318 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:20.883248 (+ 4930us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:20.883289 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:20.886878 (+ 3589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:20.890324 (+ 3446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:20.890334 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:20.890362 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:20.890910 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:20.891290 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:20.891295 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:20.891299 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:20.891536 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:20.892077 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:20.892086 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:20.892514 (+ 428us) client_negotiation.cc:769] Sending connection context
0619 02:50:20.892804 (+ 290us) client_negotiation.cc:240] Negotiation successful
0619 02:50:20.893408 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":352,"threads_started":1}
I20260619 02:50:20.908707 20302 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:37053
I20260619 02:50:20.908723 20436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:37053 every 8 connection(s)
I20260619 02:50:20.910193 20302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:50:20.912377 20437 heartbeater.cc:347] Connected to a master server at 127.18.134.62:39567
I20260619 02:50:20.912587 20437 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:20.913223 20437 heartbeater.cc:511] Master 127.18.134.62:39567 requested a full tablet report, sending...
I20260619 02:50:20.914691 20092 ts_manager.cc:194] Registered new tserver with Master: 2f694c40a66c412b879c04f7c183cfb5 (127.18.134.2:37053)
I20260619 02:50:20.915781 20092 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.2@KRBTEST.COM'} at 127.18.134.2:44361
I20260619 02:50:20.916565 20140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:20.901907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:39043 (local address 127.18.134.62:39567)
0619 02:50:20.902105 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:20.902111 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:20.903203 (+ 1092us) server_negotiation.cc:407] Connection header received
0619 02:50:20.903525 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:20.903531 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:20.903628 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:20.903700 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:20.905270 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:20.906445 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:20.908191 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:20.908838 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:20.908963 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:20.911021 (+ 2058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:20.911028 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:20.911032 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:20.911073 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:20.913888 (+ 2815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:20.914599 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:20.914604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:20.914607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:20.914780 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:20.915151 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:20.915159 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:20.915162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:20.915583 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:20.915716 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:20.916112 (+ 396us) server_negotiation.cc:299] Negotiation successful
0619 02:50:20.916312 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":67}
I20260619 02:50:20.916769 20418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:20.901959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39567 (local address 127.18.134.2:39043)
0619 02:50:20.903073 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:20.903099 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:20.903335 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:20.903921 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:20.903926 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:20.904599 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:20.905067 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:20.905083 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:20.906588 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:20.906595 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:20.907993 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:20.908017 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:20.909029 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:20.909036 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:20.909217 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:20.909855 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:20.909872 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:20.910786 (+ 914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:20.914010 (+ 3224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:20.914014 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:20.914017 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:20.914485 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:20.914889 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:20.914893 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:20.914895 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:20.915038 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:20.915677 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:20.915681 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:20.915852 (+ 171us) client_negotiation.cc:769] Sending connection context
0619 02:50:20.916000 (+ 148us) client_negotiation.cc:240] Negotiation successful
0619 02:50:20.916329 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":946,"thread_start_us":806,"threads_started":1}
I20260619 02:50:20.919646 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 20302
I20260619 02:50:20.919845 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance
I20260619 02:50:20.920393 18968 external_mini_cluster.cc:1428] Setting key 6c0f5e33b2db1621666e23cc77364f18
WARNING: no policy specified for kudu/127.18.134.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
I20260619 02:50:20.956139 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:0
--local_ip_for_outbound_sockets=127.18.134.3
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:39567
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:44663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:21.271881 20441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:21.272147 20441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:21.272240 20441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:21.279307 20441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:21.279429 20441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:21.279512 20441 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:50:21.279590 20441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:50:21.289196 20441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:44663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.134.62:39567
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:21.290468 20441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:21.291810 20441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:21.304651 20446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:21.304584 20447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:21.304613 20449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:21.305156 20441 server_base.cc:1034] running on GCE node
I20260619 02:50:21.305786 20441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:21.306743 20441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:21.307986 20441 hybrid_clock.cc:648] HybridClock initialized: now 1781837421307907 us; error 50 us; skew 500 ppm
I20260619 02:50:21.308306 20441 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:21 dist-test-slave-9mr0 krb5kdc[20047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837421, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:21.311434 20441 init.cc:377] Logged in from keytab as kudu/127.18.134.3@KRBTEST.COM (short username kudu)
I20260619 02:50:21.312891 20441 webserver.cc:460] Webserver started at http://127.18.134.3:43489/ using document root <none> and password file <none>
I20260619 02:50:21.313540 20441 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:21.313621 20441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:21.313885 20441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:21.316864 20441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-2/data/instance:
uuid: "383629b02ca34692b50370b28558e73e"
format_stamp: "Formatted at 2026-06-19 02:50:21 on dist-test-slave-9mr0"
server_key: "5965cab5bc8e4271d3658f16d87bb03d"
server_key_iv: "79c3e04b8a7f36ecdeae15893a1f71e9"
server_key_version: "encryptionkey@0"
I20260619 02:50:21.317589 20441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance:
uuid: "383629b02ca34692b50370b28558e73e"
format_stamp: "Formatted at 2026-06-19 02:50:21 on dist-test-slave-9mr0"
server_key: "5965cab5bc8e4271d3658f16d87bb03d"
server_key_iv: "79c3e04b8a7f36ecdeae15893a1f71e9"
server_key_version: "encryptionkey@0"
I20260619 02:50:21.322897 20441 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260619 02:50:21.326297 20456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:21.327934 20441 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:50:21.328086 20441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "383629b02ca34692b50370b28558e73e"
format_stamp: "Formatted at 2026-06-19 02:50:21 on dist-test-slave-9mr0"
server_key: "5965cab5bc8e4271d3658f16d87bb03d"
server_key_iv: "79c3e04b8a7f36ecdeae15893a1f71e9"
server_key_version: "encryptionkey@0"
I20260619 02:50:21.328310 20441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:21.353752 20441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:21.357831 20441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:21.358080 20441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:21.360603 20441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:21.360694 20441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:21.360826 20441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:21.360881 20441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:50:21 dist-test-slave-9mr0 krb5kdc[20047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837421, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:21.383426 20140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:21.361288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:60001 (local address 127.18.134.62:39567)
0619 02:50:21.361455 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:21.361462 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:21.362939 (+ 1477us) server_negotiation.cc:407] Connection header received
0619 02:50:21.365363 (+ 2424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:21.365369 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:21.365459 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:21.365566 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:21.367626 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.368558 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.371432 (+ 2874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.372014 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.372149 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:21.376973 (+ 4824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:21.376978 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:21.376982 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:21.377016 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:21.379784 (+ 2768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:21.380711 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:21.380719 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:21.380723 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:21.380918 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:21.381388 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:21.381392 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:21.381394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:21.381653 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:21.381781 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:21.382777 (+ 996us) server_negotiation.cc:299] Negotiation successful
0619 02:50:21.383042 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 02:50:21.384629 20465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:21.361911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39567 (local address 127.18.134.3:60001)
0619 02:50:21.362764 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:21.362810 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:21.364981 (+ 2171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:21.365853 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:21.365865 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:21.366550 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:21.367454 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:21.367477 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.368726 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.368732 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:21.371247 (+ 2515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:21.371267 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.372524 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.372531 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:21.372794 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:21.373599 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:21.373638 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:21.376652 (+ 3014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:21.379927 (+ 3275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:21.379941 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:21.379960 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:21.380557 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:21.381038 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:21.381042 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:21.381044 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:21.381266 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:21.381773 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:21.381782 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:21.382262 (+ 480us) client_negotiation.cc:769] Sending connection context
0619 02:50:21.382660 (+ 398us) client_negotiation.cc:240] Negotiation successful
0619 02:50:21.383225 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":329,"threads_started":1}
I20260619 02:50:21.398972 20441 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:40999
I20260619 02:50:21.399055 20575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:40999 every 8 connection(s)
I20260619 02:50:21.400492 20441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:50:21.402771 20576 heartbeater.cc:347] Connected to a master server at 127.18.134.62:39567
I20260619 02:50:21.402988 20576 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:21.403543 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 20441
I20260619 02:50:21.403712 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance
I20260619 02:50:21.403795 20576 heartbeater.cc:511] Master 127.18.134.62:39567 requested a full tablet report, sending...
I20260619 02:50:21.404186 18968 external_mini_cluster.cc:1428] Setting key 734fe09f96a4685bf94fa53cf2519a17
I20260619 02:50:21.405402 20092 ts_manager.cc:194] Registered new tserver with Master: 383629b02ca34692b50370b28558e73e (127.18.134.3:40999)
I20260619 02:50:21.406488 20092 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.3@KRBTEST.COM'} at 127.18.134.3:60001
I20260619 02:50:21.407495 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:50:21.411084 20140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:21.397159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:41333 (local address 127.18.134.62:39567)
0619 02:50:21.397355 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:21.397362 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:21.397901 (+ 539us) server_negotiation.cc:407] Connection header received
0619 02:50:21.398156 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:21.398163 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:21.398250 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:21.398319 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:21.399563 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.400609 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.402417 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.402993 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.403121 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:21.405217 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:21.405224 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:21.405228 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:21.405268 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:21.408282 (+ 3014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:21.409172 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:21.409177 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:21.409181 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:21.409364 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:21.409700 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:21.409705 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:21.409709 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:21.409986 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:21.410111 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:21.410503 (+ 392us) server_negotiation.cc:299] Negotiation successful
0619 02:50:21.410720 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 02:50:21.411128 20571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:21.397175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39567 (local address 127.18.134.3:41333)
0619 02:50:21.397789 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:21.397809 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:21.397974 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:21.398512 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:21.398515 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:21.399099 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:21.399415 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:21.399425 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.400784 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.400790 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:21.402239 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:21.402251 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.403175 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.403181 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:21.403363 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:21.404011 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:21.404031 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:21.405075 (+ 1044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:21.408417 (+ 3342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:21.408423 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:21.408427 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:21.409054 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:21.409466 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:21.409472 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:21.409476 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:21.409602 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:21.410091 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:21.410095 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:21.410247 (+ 152us) client_negotiation.cc:769] Sending connection context
0619 02:50:21.410376 (+ 129us) client_negotiation.cc:240] Negotiation successful
0619 02:50:21.410720 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":327,"threads_started":1}
Jun 19 02:50:21 dist-test-slave-9mr0 krb5kdc[20047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837421, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
I20260619 02:50:21.439495 20298 heartbeater.cc:503] Master 127.18.134.62:39567 was elected leader, sending a full tablet report...
Jun 19 02:50:21 dist-test-slave-9mr0 krb5kdc[20047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837421, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:21.450551 20140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:21.437365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39456 (local address 127.18.134.62:39567)
0619 02:50:21.437594 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:21.437600 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:21.437764 (+ 164us) server_negotiation.cc:407] Connection header received
0619 02:50:21.437948 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:21.437954 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:21.438021 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:21.438101 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:21.439491 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.440450 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.441788 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.442190 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.442283 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:21.445758 (+ 3475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:21.445762 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:21.445766 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:21.445798 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:21.448287 (+ 2489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:21.448898 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:21.448902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:21.448904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:21.449034 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:21.449400 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:21.449403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:21.449405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:21.449654 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:21.449820 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:21.450102 (+ 282us) server_negotiation.cc:299] Negotiation successful
0619 02:50:21.450261 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260619 02:50:21.454178 20091 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39456:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:50:21.460978 20091 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:50:21.499195 20590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:21.473256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:37053 (local address 127.0.0.1:40156)
0619 02:50:21.491047 (+ 17791us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:21.491068 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:21.491211 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:21.492210 (+ 999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:21.492218 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:21.492254 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:21.492727 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:21.492737 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.494327 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.494333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:21.496453 (+ 2120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:21.496464 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.498141 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.498147 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:21.498344 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:21.498426 (+ 82us) client_negotiation.cc:769] Sending connection context
0619 02:50:21.498554 (+ 128us) client_negotiation.cc:240] Negotiation successful
0619 02:50:21.498786 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17654,"mutex_wait_us":48,"spinlock_wait_cycles":2304,"thread_start_us":17116,"threads_started":1}
I20260619 02:50:21.499930 20594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:21.473708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40156 (local address 127.18.134.2:37053)
0619 02:50:21.490178 (+ 16470us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:21.490188 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:21.491329 (+ 1141us) server_negotiation.cc:407] Connection header received
0619 02:50:21.491459 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:21.491468 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:21.491770 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:21.491968 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:21.492909 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.494211 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.496658 (+ 2447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.497783 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.497906 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:21.498126 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:21.499472 (+ 1346us) server_negotiation.cc:299] Negotiation successful
0619 02:50:21.499652 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16288,"thread_start_us":16160,"threads_started":1}
I20260619 02:50:21.500546 20593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:21.472739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:40999 (local address 127.0.0.1:51030)
0619 02:50:21.490653 (+ 17914us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:21.490685 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:21.490818 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:21.492378 (+ 1560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:21.492384 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:21.492415 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:21.492951 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:21.492963 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.494598 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.494604 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:21.496972 (+ 2368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:21.496986 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.499603 (+ 2617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.499610 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:21.499767 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:21.499799 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 02:50:21.499913 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 02:50:21.500168 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17762,"mutex_wait_us":60,"thread_start_us":18028,"threads_started":1}
I20260619 02:50:21.501230 20589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:21.472570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:36831 (local address 127.0.0.1:39154)
0619 02:50:21.489246 (+ 16676us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:21.489311 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:21.489512 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:21.493774 (+ 4262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:21.493780 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:21.493804 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:21.494100 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:21.494111 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.495886 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.495892 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:21.497900 (+ 2008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:21.497916 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.500469 (+ 2553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.500475 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:21.500651 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:21.500681 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 02:50:21.500760 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 02:50:21.500885 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16381,"mutex_wait_us":21,"thread_start_us":14525,"threads_started":1}
I20260619 02:50:21.501313 20592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:21.473162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51030 (local address 127.18.134.3:40999)
0619 02:50:21.491462 (+ 18300us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:21.491472 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:21.491507 (+ 35us) server_negotiation.cc:407] Connection header received
0619 02:50:21.491609 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:21.491618 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:21.491885 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:21.492073 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:21.493152 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.494457 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.497136 (+ 2679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.498794 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.498942 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:21.499182 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:21.500833 (+ 1651us) server_negotiation.cc:299] Negotiation successful
0619 02:50:21.501023 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18107,"thread_start_us":19193,"threads_started":1}
I20260619 02:50:21.501536 20591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:21.472849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39154 (local address 127.18.134.1:36831)
0619 02:50:21.493223 (+ 20374us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:21.493231 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:21.493256 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:50:21.493345 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:21.493353 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:21.493575 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:21.493767 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:21.494277 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.495732 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.498069 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.499997 (+ 1928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.500109 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:21.500304 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:21.501101 (+ 797us) server_negotiation.cc:299] Negotiation successful
0619 02:50:21.501278 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20228,"thread_start_us":20107,"threads_started":1}
I20260619 02:50:21.503743 20367 tablet_service.cc:1467] Processing CreateTablet for tablet e49dd7bdb8c142f3a3ffad26622f6690 (DEFAULT_TABLE table=test-table [id=931c4eb8856f46f8b8547a97b10f5e51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:50:21.504432 20228 tablet_service.cc:1467] Processing CreateTablet for tablet e49dd7bdb8c142f3a3ffad26622f6690 (DEFAULT_TABLE table=test-table [id=931c4eb8856f46f8b8547a97b10f5e51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:50:21.504796 20506 tablet_service.cc:1467] Processing CreateTablet for tablet e49dd7bdb8c142f3a3ffad26622f6690 (DEFAULT_TABLE table=test-table [id=931c4eb8856f46f8b8547a97b10f5e51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:50:21.505744 20228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e49dd7bdb8c142f3a3ffad26622f6690. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:21.505777 20367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e49dd7bdb8c142f3a3ffad26622f6690. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:21.505970 20506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e49dd7bdb8c142f3a3ffad26622f6690. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:21.517213 20595 tablet_bootstrap.cc:492] T e49dd7bdb8c142f3a3ffad26622f6690 P 0c2b46b6e18547e99f0c75cd3eb9e09d: Bootstrap starting.
I20260619 02:50:21.520498 20597 tablet_bootstrap.cc:492] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5: Bootstrap starting.
I20260619 02:50:21.521363 20595 tablet_bootstrap.cc:654] T e49dd7bdb8c142f3a3ffad26622f6690 P 0c2b46b6e18547e99f0c75cd3eb9e09d: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:21.522744 20595 log.cc:824] T e49dd7bdb8c142f3a3ffad26622f6690 P 0c2b46b6e18547e99f0c75cd3eb9e09d: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:21.523975 20597 tablet_bootstrap.cc:654] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:21.524106 20596 tablet_bootstrap.cc:492] T e49dd7bdb8c142f3a3ffad26622f6690 P 383629b02ca34692b50370b28558e73e: Bootstrap starting.
I20260619 02:50:21.525224 20597 log.cc:824] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:21.526089 20595 tablet_bootstrap.cc:492] T e49dd7bdb8c142f3a3ffad26622f6690 P 0c2b46b6e18547e99f0c75cd3eb9e09d: No bootstrap required, opened a new log
I20260619 02:50:21.526368 20595 ts_tablet_manager.cc:1397] T e49dd7bdb8c142f3a3ffad26622f6690 P 0c2b46b6e18547e99f0c75cd3eb9e09d: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260619 02:50:21.527508 20596 tablet_bootstrap.cc:654] T e49dd7bdb8c142f3a3ffad26622f6690 P 383629b02ca34692b50370b28558e73e: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:21.528388 20597 tablet_bootstrap.cc:492] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5: No bootstrap required, opened a new log
I20260619 02:50:21.528651 20597 ts_tablet_manager.cc:1397] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260619 02:50:21.528777 20596 log.cc:824] T e49dd7bdb8c142f3a3ffad26622f6690 P 383629b02ca34692b50370b28558e73e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:21.531695 20596 tablet_bootstrap.cc:492] T e49dd7bdb8c142f3a3ffad26622f6690 P 383629b02ca34692b50370b28558e73e: No bootstrap required, opened a new log
I20260619 02:50:21.531996 20596 ts_tablet_manager.cc:1397] T e49dd7bdb8c142f3a3ffad26622f6690 P 383629b02ca34692b50370b28558e73e: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260619 02:50:21.535020 20595 raft_consensus.cc:348] T e49dd7bdb8c142f3a3ffad26622f6690 P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } }
I20260619 02:50:21.535391 20595 raft_consensus.cc:374] T e49dd7bdb8c142f3a3ffad26622f6690 P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:21.535521 20595 raft_consensus.cc:729] T e49dd7bdb8c142f3a3ffad26622f6690 P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c2b46b6e18547e99f0c75cd3eb9e09d, State: Initialized, Role: FOLLOWER
I20260619 02:50:21.536108 20595 consensus_queue.cc:260] T e49dd7bdb8c142f3a3ffad26622f6690 P 0c2b46b6e18547e99f0c75cd3eb9e09d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } }
I20260619 02:50:21.537827 20595 ts_tablet_manager.cc:1428] T e49dd7bdb8c142f3a3ffad26622f6690 P 0c2b46b6e18547e99f0c75cd3eb9e09d: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260619 02:50:21.539373 20597 raft_consensus.cc:348] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } }
I20260619 02:50:21.539803 20597 raft_consensus.cc:374] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:21.539909 20597 raft_consensus.cc:729] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f694c40a66c412b879c04f7c183cfb5, State: Initialized, Role: FOLLOWER
I20260619 02:50:21.540512 20597 consensus_queue.cc:260] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } }
I20260619 02:50:21.542093 20437 heartbeater.cc:503] Master 127.18.134.62:39567 was elected leader, sending a full tablet report...
I20260619 02:50:21.542173 20597 ts_tablet_manager.cc:1428] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260619 02:50:21.542443 20596 raft_consensus.cc:348] T e49dd7bdb8c142f3a3ffad26622f6690 P 383629b02ca34692b50370b28558e73e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } }
I20260619 02:50:21.542855 20596 raft_consensus.cc:374] T e49dd7bdb8c142f3a3ffad26622f6690 P 383629b02ca34692b50370b28558e73e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:21.542950 20596 raft_consensus.cc:729] T e49dd7bdb8c142f3a3ffad26622f6690 P 383629b02ca34692b50370b28558e73e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 383629b02ca34692b50370b28558e73e, State: Initialized, Role: FOLLOWER
I20260619 02:50:21.543515 20596 consensus_queue.cc:260] T e49dd7bdb8c142f3a3ffad26622f6690 P 383629b02ca34692b50370b28558e73e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } }
I20260619 02:50:21.545071 20576 heartbeater.cc:503] Master 127.18.134.62:39567 was elected leader, sending a full tablet report...
I20260619 02:50:21.545673 20596 ts_tablet_manager.cc:1428] T e49dd7bdb8c142f3a3ffad26622f6690 P 383629b02ca34692b50370b28558e73e: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260619 02:50:21.587060 20602 raft_consensus.cc:482] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:50:21.587397 20602 raft_consensus.cc:504] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } }
I20260619 02:50:21.589126 20602 leader_election.cc:288] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c2b46b6e18547e99f0c75cd3eb9e09d (127.18.134.1:36831), 383629b02ca34692b50370b28558e73e (127.18.134.3:40999)
I20260619 02:50:21.595470 20592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:21.589736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:38237 (local address 127.18.134.3:40999)
0619 02:50:21.590123 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:21.590129 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:21.590672 (+ 543us) server_negotiation.cc:407] Connection header received
0619 02:50:21.590864 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:21.590868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:21.590926 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:21.590996 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:21.591469 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.592125 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.593819 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.594577 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.594771 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:21.594845 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:21.595095 (+ 250us) server_negotiation.cc:299] Negotiation successful
0619 02:50:21.595227 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261}
I20260619 02:50:21.595665 20605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:21.589510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:40999 (local address 127.18.134.2:38237)
0619 02:50:21.590561 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:21.590574 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:21.590731 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:21.591025 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:21.591029 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:21.591051 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:21.591338 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:21.591346 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.592258 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.592264 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:21.593675 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:21.593686 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.594802 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.594807 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:21.594953 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:21.594976 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 02:50:21.595067 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 02:50:21.595204 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":950,"mutex_wait_us":1,"thread_start_us":763,"threads_started":1}
I20260619 02:50:21.595723 20591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:21.589614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:48491 (local address 127.18.134.1:36831)
0619 02:50:21.589970 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:21.589978 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:21.590240 (+ 262us) server_negotiation.cc:407] Connection header received
0619 02:50:21.590445 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:21.590450 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:21.590507 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:21.590580 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:21.591097 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.591804 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.593675 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.594881 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.595019 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:21.595090 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:21.595379 (+ 289us) server_negotiation.cc:299] Negotiation successful
0619 02:50:21.595508 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202}
I20260619 02:50:21.596215 20604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:21.589427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:36831 (local address 127.18.134.2:48491)
0619 02:50:21.590124 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:21.590143 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:21.590303 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:21.590611 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:21.590616 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:21.590673 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:21.590972 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:21.590980 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.591928 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.591933 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:21.593525 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:21.593536 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.595080 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.595085 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:21.595233 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:21.595263 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 02:50:21.595348 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 02:50:21.595470 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"mutex_wait_us":79,"thread_start_us":495,"threads_started":1}
I20260619 02:50:21.596834 20526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e49dd7bdb8c142f3a3ffad26622f6690" candidate_uuid: "2f694c40a66c412b879c04f7c183cfb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "383629b02ca34692b50370b28558e73e" is_pre_election: true
I20260619 02:50:21.597124 20248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e49dd7bdb8c142f3a3ffad26622f6690" candidate_uuid: "2f694c40a66c412b879c04f7c183cfb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" is_pre_election: true
I20260619 02:50:21.597373 20526 raft_consensus.cc:2454] T e49dd7bdb8c142f3a3ffad26622f6690 P 383629b02ca34692b50370b28558e73e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f694c40a66c412b879c04f7c183cfb5 in term 0.
I20260619 02:50:21.597642 20248 raft_consensus.cc:2454] T e49dd7bdb8c142f3a3ffad26622f6690 P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f694c40a66c412b879c04f7c183cfb5 in term 0.
I20260619 02:50:21.598069 20318 leader_election.cc:302] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f694c40a66c412b879c04f7c183cfb5, 383629b02ca34692b50370b28558e73e; no voters:
I20260619 02:50:21.598479 20602 raft_consensus.cc:2781] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:50:21.598603 20602 raft_consensus.cc:482] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:50:21.598726 20602 raft_consensus.cc:3037] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:21.600972 20602 raft_consensus.cc:504] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } }
I20260619 02:50:21.601583 20602 leader_election.cc:288] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5 [CANDIDATE]: Term 1 election: Requested vote from peers 0c2b46b6e18547e99f0c75cd3eb9e09d (127.18.134.1:36831), 383629b02ca34692b50370b28558e73e (127.18.134.3:40999)
I20260619 02:50:21.601934 20248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e49dd7bdb8c142f3a3ffad26622f6690" candidate_uuid: "2f694c40a66c412b879c04f7c183cfb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d"
I20260619 02:50:21.602025 20526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e49dd7bdb8c142f3a3ffad26622f6690" candidate_uuid: "2f694c40a66c412b879c04f7c183cfb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "383629b02ca34692b50370b28558e73e"
I20260619 02:50:21.602105 20248 raft_consensus.cc:3037] T e49dd7bdb8c142f3a3ffad26622f6690 P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:21.602214 20526 raft_consensus.cc:3037] T e49dd7bdb8c142f3a3ffad26622f6690 P 383629b02ca34692b50370b28558e73e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:21.604274 20248 raft_consensus.cc:2454] T e49dd7bdb8c142f3a3ffad26622f6690 P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f694c40a66c412b879c04f7c183cfb5 in term 1.
I20260619 02:50:21.604374 20526 raft_consensus.cc:2454] T e49dd7bdb8c142f3a3ffad26622f6690 P 383629b02ca34692b50370b28558e73e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f694c40a66c412b879c04f7c183cfb5 in term 1.
I20260619 02:50:21.604661 20320 leader_election.cc:302] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c2b46b6e18547e99f0c75cd3eb9e09d, 2f694c40a66c412b879c04f7c183cfb5; no voters:
I20260619 02:50:21.604959 20602 raft_consensus.cc:2781] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:50:21.605326 20602 raft_consensus.cc:686] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5 [term 1 LEADER]: Becoming Leader. State: Replica: 2f694c40a66c412b879c04f7c183cfb5, State: Running, Role: LEADER
I20260619 02:50:21.605796 20602 consensus_queue.cc:237] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } }
I20260619 02:50:21.610468 20092 catalog_manager.cc:5495] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f694c40a66c412b879c04f7c183cfb5 (127.18.134.2). New cstate: current_term: 1 leader_uuid: "2f694c40a66c412b879c04f7c183cfb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } health_report { overall_health: HEALTHY } } }
W20260619 02:50:21.652236 20577 tablet.cc:2369] T e49dd7bdb8c142f3a3ffad26622f6690 P 383629b02ca34692b50370b28558e73e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:50:21.659942 20594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:21.654729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40170 (local address 127.18.134.2:37053)
0619 02:50:21.654948 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:21.654955 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:21.655073 (+ 118us) server_negotiation.cc:407] Connection header received
0619 02:50:21.655309 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:21.655313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:21.655394 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:21.655519 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:50:21.655802 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.656519 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.657868 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.658305 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.658405 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:21.658829 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:50:21.659360 (+ 531us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:50:21.659465 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:21.659576 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 02:50:21.659644 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260619 02:50:21.662182 20438 tablet.cc:2369] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:50:21.668015 20248 raft_consensus.cc:1261] T e49dd7bdb8c142f3a3ffad26622f6690 P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 1 FOLLOWER]: Refusing update from remote peer 2f694c40a66c412b879c04f7c183cfb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:50:21.668535 20526 raft_consensus.cc:1261] T e49dd7bdb8c142f3a3ffad26622f6690 P 383629b02ca34692b50370b28558e73e [term 1 FOLLOWER]: Refusing update from remote peer 2f694c40a66c412b879c04f7c183cfb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:50:21.668931 20606 consensus_queue.cc:1035] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:50:21.669415 20602 consensus_queue.cc:1035] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:50:21.681263 20610 mvcc.cc:205] Tried to move back new op lower bound from 7298406079142518784 to 7298406078901481472. Current Snapshot: MvccSnapshot[applied={T|T < 7298406079142518784}]
I20260619 02:50:21.699745 20092 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:50:21.700014 20092 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:50:21.703459 20092 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a6c62c7df18c4c549706d19cd35f702a: Sending DeleteTablet for 3 replicas of tablet e49dd7bdb8c142f3a3ffad26622f6690
I20260619 02:50:21.704499 20506 tablet_service.cc:1514] Processing DeleteTablet for tablet e49dd7bdb8c142f3a3ffad26622f6690 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:50:21 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:51030
I20260619 02:50:21.704511 20367 tablet_service.cc:1514] Processing DeleteTablet for tablet e49dd7bdb8c142f3a3ffad26622f6690 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:50:21 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:40156
I20260619 02:50:21.704692 20228 tablet_service.cc:1514] Processing DeleteTablet for tablet e49dd7bdb8c142f3a3ffad26622f6690 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:50:21 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:39154
I20260619 02:50:21.707191 20092 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39456:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:50:21.708388 20092 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:50:21.714869 20621 tablet_replica.cc:330] T e49dd7bdb8c142f3a3ffad26622f6690 P 0c2b46b6e18547e99f0c75cd3eb9e09d: stopping tablet replica
I20260619 02:50:21.715329 20621 raft_consensus.cc:2229] T e49dd7bdb8c142f3a3ffad26622f6690 P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:50:21.715874 20621 raft_consensus.cc:2258] T e49dd7bdb8c142f3a3ffad26622f6690 P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:50:21.716753 20228 tablet_service.cc:1467] Processing CreateTablet for tablet 46259ac9881c42ecb1e60fbe76f13d58 (DEFAULT_TABLE table=test-table [id=8e37c48a3b4d4362852bcf1104c5d6c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:50:21.717278 20228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46259ac9881c42ecb1e60fbe76f13d58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:21.717207 20505 tablet_service.cc:1467] Processing CreateTablet for tablet 46259ac9881c42ecb1e60fbe76f13d58 (DEFAULT_TABLE table=test-table [id=8e37c48a3b4d4362852bcf1104c5d6c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:50:21.718271 20505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46259ac9881c42ecb1e60fbe76f13d58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:21.718902 20621 ts_tablet_manager.cc:1905] T e49dd7bdb8c142f3a3ffad26622f6690 P 0c2b46b6e18547e99f0c75cd3eb9e09d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:50:21.719520 20366 tablet_service.cc:1467] Processing CreateTablet for tablet 46259ac9881c42ecb1e60fbe76f13d58 (DEFAULT_TABLE table=test-table [id=8e37c48a3b4d4362852bcf1104c5d6c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:50:21.720103 20366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46259ac9881c42ecb1e60fbe76f13d58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:21.725471 20621 ts_tablet_manager.cc:1918] T e49dd7bdb8c142f3a3ffad26622f6690 P 0c2b46b6e18547e99f0c75cd3eb9e09d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:50:21.725620 20621 log.cc:1196] T e49dd7bdb8c142f3a3ffad26622f6690 P 0c2b46b6e18547e99f0c75cd3eb9e09d: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-0/wal/wals/e49dd7bdb8c142f3a3ffad26622f6690
I20260619 02:50:21.726130 20621 ts_tablet_manager.cc:1939] T e49dd7bdb8c142f3a3ffad26622f6690 P 0c2b46b6e18547e99f0c75cd3eb9e09d: Deleting consensus metadata
I20260619 02:50:21.730265 20595 tablet_bootstrap.cc:492] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d: Bootstrap starting.
I20260619 02:50:21.731796 20595 tablet_bootstrap.cc:654] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:21.732144 20597 tablet_bootstrap.cc:492] T 46259ac9881c42ecb1e60fbe76f13d58 P 2f694c40a66c412b879c04f7c183cfb5: Bootstrap starting.
I20260619 02:50:21.733110 20078 catalog_manager.cc:4841] TS 0c2b46b6e18547e99f0c75cd3eb9e09d (127.18.134.1:36831): tablet e49dd7bdb8c142f3a3ffad26622f6690 (table test-table [id=931c4eb8856f46f8b8547a97b10f5e51]) successfully deleted
I20260619 02:50:21.733820 20597 tablet_bootstrap.cc:654] T 46259ac9881c42ecb1e60fbe76f13d58 P 2f694c40a66c412b879c04f7c183cfb5: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:21.739542 20596 tablet_bootstrap.cc:492] T 46259ac9881c42ecb1e60fbe76f13d58 P 383629b02ca34692b50370b28558e73e: Bootstrap starting.
I20260619 02:50:21.741931 20596 tablet_bootstrap.cc:654] T 46259ac9881c42ecb1e60fbe76f13d58 P 383629b02ca34692b50370b28558e73e: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:21.747072 20620 tablet_replica.cc:330] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5: stopping tablet replica
I20260619 02:50:21.748302 20620 raft_consensus.cc:2229] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:50:21.749715 20620 raft_consensus.cc:2258] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:50:21.752954 20620 ts_tablet_manager.cc:1905] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:50:21.754331 20619 tablet_replica.cc:330] T e49dd7bdb8c142f3a3ffad26622f6690 P 383629b02ca34692b50370b28558e73e: stopping tablet replica
I20260619 02:50:21.755667 20595 tablet_bootstrap.cc:492] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d: No bootstrap required, opened a new log
I20260619 02:50:21.755867 20596 tablet_bootstrap.cc:492] T 46259ac9881c42ecb1e60fbe76f13d58 P 383629b02ca34692b50370b28558e73e: No bootstrap required, opened a new log
I20260619 02:50:21.755901 20595 ts_tablet_manager.cc:1397] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d: Time spent bootstrapping tablet: real 0.026s user 0.005s sys 0.009s
I20260619 02:50:21.755934 20619 raft_consensus.cc:2229] T e49dd7bdb8c142f3a3ffad26622f6690 P 383629b02ca34692b50370b28558e73e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:50:21.756027 20596 ts_tablet_manager.cc:1397] T 46259ac9881c42ecb1e60fbe76f13d58 P 383629b02ca34692b50370b28558e73e: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.004s
I20260619 02:50:21.756916 20595 raft_consensus.cc:348] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } }
I20260619 02:50:21.757210 20595 raft_consensus.cc:374] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:21.757364 20595 raft_consensus.cc:729] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c2b46b6e18547e99f0c75cd3eb9e09d, State: Initialized, Role: FOLLOWER
I20260619 02:50:21.757736 20595 consensus_queue.cc:260] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } }
I20260619 02:50:21.758209 20597 tablet_bootstrap.cc:492] T 46259ac9881c42ecb1e60fbe76f13d58 P 2f694c40a66c412b879c04f7c183cfb5: No bootstrap required, opened a new log
I20260619 02:50:21.758354 20597 ts_tablet_manager.cc:1397] T 46259ac9881c42ecb1e60fbe76f13d58 P 2f694c40a66c412b879c04f7c183cfb5: Time spent bootstrapping tablet: real 0.026s user 0.009s sys 0.003s
I20260619 02:50:21.759199 20597 raft_consensus.cc:348] T 46259ac9881c42ecb1e60fbe76f13d58 P 2f694c40a66c412b879c04f7c183cfb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } }
I20260619 02:50:21.759423 20597 raft_consensus.cc:374] T 46259ac9881c42ecb1e60fbe76f13d58 P 2f694c40a66c412b879c04f7c183cfb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:21.759526 20597 raft_consensus.cc:729] T 46259ac9881c42ecb1e60fbe76f13d58 P 2f694c40a66c412b879c04f7c183cfb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f694c40a66c412b879c04f7c183cfb5, State: Initialized, Role: FOLLOWER
I20260619 02:50:21.759796 20597 consensus_queue.cc:260] T 46259ac9881c42ecb1e60fbe76f13d58 P 2f694c40a66c412b879c04f7c183cfb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } }
I20260619 02:50:21.760346 20597 ts_tablet_manager.cc:1428] T 46259ac9881c42ecb1e60fbe76f13d58 P 2f694c40a66c412b879c04f7c183cfb5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:50:21.760599 20596 raft_consensus.cc:348] T 46259ac9881c42ecb1e60fbe76f13d58 P 383629b02ca34692b50370b28558e73e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } }
I20260619 02:50:21.760836 20596 raft_consensus.cc:374] T 46259ac9881c42ecb1e60fbe76f13d58 P 383629b02ca34692b50370b28558e73e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:21.760946 20596 raft_consensus.cc:729] T 46259ac9881c42ecb1e60fbe76f13d58 P 383629b02ca34692b50370b28558e73e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 383629b02ca34692b50370b28558e73e, State: Initialized, Role: FOLLOWER
I20260619 02:50:21.761135 20595 ts_tablet_manager.cc:1428] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260619 02:50:21.761266 20596 consensus_queue.cc:260] T 46259ac9881c42ecb1e60fbe76f13d58 P 383629b02ca34692b50370b28558e73e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } }
I20260619 02:50:21.761790 20596 ts_tablet_manager.cc:1428] T 46259ac9881c42ecb1e60fbe76f13d58 P 383629b02ca34692b50370b28558e73e: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260619 02:50:21.761977 20619 raft_consensus.cc:2258] T e49dd7bdb8c142f3a3ffad26622f6690 P 383629b02ca34692b50370b28558e73e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:50:21.764581 20620 ts_tablet_manager.cc:1918] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:50:21.764743 20620 log.cc:1196] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-1/wal/wals/e49dd7bdb8c142f3a3ffad26622f6690
I20260619 02:50:21.765280 20620 ts_tablet_manager.cc:1939] T e49dd7bdb8c142f3a3ffad26622f6690 P 2f694c40a66c412b879c04f7c183cfb5: Deleting consensus metadata
I20260619 02:50:21.766577 20619 ts_tablet_manager.cc:1905] T e49dd7bdb8c142f3a3ffad26622f6690 P 383629b02ca34692b50370b28558e73e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:50:21.766957 20077 catalog_manager.cc:4841] TS 2f694c40a66c412b879c04f7c183cfb5 (127.18.134.2:37053): tablet e49dd7bdb8c142f3a3ffad26622f6690 (table test-table [id=931c4eb8856f46f8b8547a97b10f5e51]) successfully deleted
I20260619 02:50:21.771456 20619 ts_tablet_manager.cc:1918] T e49dd7bdb8c142f3a3ffad26622f6690 P 383629b02ca34692b50370b28558e73e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:50:21.771598 20619 log.cc:1196] T e49dd7bdb8c142f3a3ffad26622f6690 P 383629b02ca34692b50370b28558e73e: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-2/wal/wals/e49dd7bdb8c142f3a3ffad26622f6690
I20260619 02:50:21.771991 20619 ts_tablet_manager.cc:1939] T e49dd7bdb8c142f3a3ffad26622f6690 P 383629b02ca34692b50370b28558e73e: Deleting consensus metadata
I20260619 02:50:21.773460 20076 catalog_manager.cc:4841] TS 383629b02ca34692b50370b28558e73e (127.18.134.3:40999): tablet e49dd7bdb8c142f3a3ffad26622f6690 (table test-table [id=931c4eb8856f46f8b8547a97b10f5e51]) successfully deleted
I20260619 02:50:21.797120 20601 raft_consensus.cc:482] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:50:21.797447 20601 raft_consensus.cc:504] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } }
I20260619 02:50:21.799610 20601 leader_election.cc:288] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f694c40a66c412b879c04f7c183cfb5 (127.18.134.2:37053), 383629b02ca34692b50370b28558e73e (127.18.134.3:40999)
I20260619 02:50:21.806619 20592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:21.800348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:37035 (local address 127.18.134.3:40999)
0619 02:50:21.800500 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:21.800507 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:21.801231 (+ 724us) server_negotiation.cc:407] Connection header received
0619 02:50:21.801386 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:21.801391 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:21.801466 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:21.801546 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:21.802261 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.803269 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.804986 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.805708 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.805859 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:21.805934 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:21.806250 (+ 316us) server_negotiation.cc:299] Negotiation successful
0619 02:50:21.806375 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 02:50:21.806744 20625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:21.799912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:40999 (local address 127.18.134.1:37035)
0619 02:50:21.801123 (+ 1211us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:21.801139 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:21.801249 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:21.801717 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:21.801722 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:21.801750 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:21.802083 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:21.802094 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.803391 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.803395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:21.804835 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:21.804847 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.805959 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.805966 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:21.806107 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:21.806131 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 02:50:21.806210 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 02:50:21.806345 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1115,"mutex_wait_us":5,"thread_start_us":608,"threads_started":1}
I20260619 02:50:21.807634 20526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46259ac9881c42ecb1e60fbe76f13d58" candidate_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "383629b02ca34692b50370b28558e73e" is_pre_election: true
I20260619 02:50:21.807935 20526 raft_consensus.cc:2454] T 46259ac9881c42ecb1e60fbe76f13d58 P 383629b02ca34692b50370b28558e73e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c2b46b6e18547e99f0c75cd3eb9e09d in term 0.
I20260619 02:50:21.808423 20594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:21.800153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:45765 (local address 127.18.134.2:37053)
0619 02:50:21.800340 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:21.800347 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:21.800902 (+ 555us) server_negotiation.cc:407] Connection header received
0619 02:50:21.801184 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:21.801190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:21.801296 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:21.801383 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:21.801844 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.802861 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.805963 (+ 3102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.807263 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.807399 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:21.807474 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:21.807989 (+ 515us) server_negotiation.cc:299] Negotiation successful
0619 02:50:21.808123 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 02:50:21.808611 20626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:21.799844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:37053 (local address 127.18.134.1:45765)
0619 02:50:21.800796 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:21.800815 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:21.801024 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:21.801394 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:21.801398 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:21.801420 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:21.801696 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:21.801705 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.803010 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.803015 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:21.805777 (+ 2762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:21.805792 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.807653 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.807659 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:21.807824 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:21.807859 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 02:50:21.807953 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 02:50:21.808087 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":808,"mutex_wait_us":56,"thread_start_us":1009,"threads_started":1}
I20260619 02:50:21.808506 20179 leader_election.cc:302] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c2b46b6e18547e99f0c75cd3eb9e09d, 383629b02ca34692b50370b28558e73e; no voters:
I20260619 02:50:21.808949 20601 raft_consensus.cc:2781] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:50:21.809051 20601 raft_consensus.cc:482] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:50:21.809132 20601 raft_consensus.cc:3037] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:21.809499 20387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46259ac9881c42ecb1e60fbe76f13d58" candidate_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f694c40a66c412b879c04f7c183cfb5" is_pre_election: true
I20260619 02:50:21.809928 20387 raft_consensus.cc:2454] T 46259ac9881c42ecb1e60fbe76f13d58 P 2f694c40a66c412b879c04f7c183cfb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c2b46b6e18547e99f0c75cd3eb9e09d in term 0.
I20260619 02:50:21.811631 20601 raft_consensus.cc:504] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } }
I20260619 02:50:21.812212 20601 leader_election.cc:288] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d [CANDIDATE]: Term 1 election: Requested vote from peers 2f694c40a66c412b879c04f7c183cfb5 (127.18.134.2:37053), 383629b02ca34692b50370b28558e73e (127.18.134.3:40999)
I20260619 02:50:21.812685 20387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46259ac9881c42ecb1e60fbe76f13d58" candidate_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f694c40a66c412b879c04f7c183cfb5"
I20260619 02:50:21.812734 20526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46259ac9881c42ecb1e60fbe76f13d58" candidate_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "383629b02ca34692b50370b28558e73e"
I20260619 02:50:21.812908 20526 raft_consensus.cc:3037] T 46259ac9881c42ecb1e60fbe76f13d58 P 383629b02ca34692b50370b28558e73e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:21.812906 20387 raft_consensus.cc:3037] T 46259ac9881c42ecb1e60fbe76f13d58 P 2f694c40a66c412b879c04f7c183cfb5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:21.815848 20526 raft_consensus.cc:2454] T 46259ac9881c42ecb1e60fbe76f13d58 P 383629b02ca34692b50370b28558e73e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c2b46b6e18547e99f0c75cd3eb9e09d in term 1.
I20260619 02:50:21.815965 20387 raft_consensus.cc:2454] T 46259ac9881c42ecb1e60fbe76f13d58 P 2f694c40a66c412b879c04f7c183cfb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c2b46b6e18547e99f0c75cd3eb9e09d in term 1.
I20260619 02:50:21.816318 20180 leader_election.cc:302] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c2b46b6e18547e99f0c75cd3eb9e09d, 2f694c40a66c412b879c04f7c183cfb5; no voters:
I20260619 02:50:21.816606 20601 raft_consensus.cc:2781] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:50:21.817047 20601 raft_consensus.cc:686] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 1 LEADER]: Becoming Leader. State: Replica: 0c2b46b6e18547e99f0c75cd3eb9e09d, State: Running, Role: LEADER
I20260619 02:50:21.817456 20601 consensus_queue.cc:237] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } }
I20260619 02:50:21.820674 20092 catalog_manager.cc:5495] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c2b46b6e18547e99f0c75cd3eb9e09d (127.18.134.1). New cstate: current_term: 1 leader_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } health_report { overall_health: UNKNOWN } } }
Jun 19 02:50:21 dist-test-slave-9mr0 krb5kdc[20047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837419, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:21.883143 20634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:21.865220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39567 (local address 127.0.0.1:39470)
0619 02:50:21.866315 (+ 1095us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:21.866342 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:21.866507 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:21.866880 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:21.866886 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:21.867846 (+ 960us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:21.868225 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:21.868238 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.869671 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.869677 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:21.871314 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:21.871329 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.872367 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.872374 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:21.872555 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:21.873369 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:21.873394 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:21.876386 (+ 2992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:21.880230 (+ 3844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:21.880244 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:21.880253 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:21.880836 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:21.881241 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:21.881247 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:21.881250 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:21.881418 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:21.882004 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:21.882013 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:21.882202 (+ 189us) client_negotiation.cc:769] Sending connection context
0619 02:50:21.882379 (+ 177us) client_negotiation.cc:240] Negotiation successful
0619 02:50:21.882707 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":358,"threads_started":1}
I20260619 02:50:21.883577 20140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:21.865163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39470 (local address 127.18.134.62:39567)
0619 02:50:21.865415 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:21.865422 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:21.866446 (+ 1024us) server_negotiation.cc:407] Connection header received
0619 02:50:21.866663 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:21.866669 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:21.866744 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:21.866833 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:21.868380 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.869530 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.871464 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:21.872160 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:21.872282 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:21.876596 (+ 4314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:21.876603 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:21.876608 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:21.876646 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:21.880086 (+ 3440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:21.880964 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:21.880970 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:21.880974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:21.881128 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:21.881535 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:21.881540 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:21.881543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:21.881871 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:21.882021 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:21.882494 (+ 473us) server_negotiation.cc:299] Negotiation successful
0619 02:50:21.882868 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":142,"server-negotiator.queue_time_us":113}
I20260619 02:50:21.888748 20092 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:39470:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 02:50:21.890100 20092 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:50:21.897291 20228 tablet_service.cc:1467] Processing CreateTablet for tablet a03a92ee8712475f9430615e968c669d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d5166a3669e641e6bb3ea480623195f1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:50:21.897625 20506 tablet_service.cc:1467] Processing CreateTablet for tablet a03a92ee8712475f9430615e968c669d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d5166a3669e641e6bb3ea480623195f1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:50:21.897922 20228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a03a92ee8712475f9430615e968c669d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:21.898195 20506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a03a92ee8712475f9430615e968c669d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:21.900866 20367 tablet_service.cc:1467] Processing CreateTablet for tablet a03a92ee8712475f9430615e968c669d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d5166a3669e641e6bb3ea480623195f1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:50:21.901408 20367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a03a92ee8712475f9430615e968c669d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:21.905966 20596 tablet_bootstrap.cc:492] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e: Bootstrap starting.
I20260619 02:50:21.907598 20596 tablet_bootstrap.cc:654] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:21.908631 20597 tablet_bootstrap.cc:492] T a03a92ee8712475f9430615e968c669d P 2f694c40a66c412b879c04f7c183cfb5: Bootstrap starting.
I20260619 02:50:21.910342 20597 tablet_bootstrap.cc:654] T a03a92ee8712475f9430615e968c669d P 2f694c40a66c412b879c04f7c183cfb5: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:21.912770 20596 tablet_bootstrap.cc:492] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e: No bootstrap required, opened a new log
I20260619 02:50:21.912941 20596 ts_tablet_manager.cc:1397] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.003s
I20260619 02:50:21.913717 20596 raft_consensus.cc:348] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } }
I20260619 02:50:21.913975 20596 raft_consensus.cc:374] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:21.913996 20595 tablet_bootstrap.cc:492] T a03a92ee8712475f9430615e968c669d P 0c2b46b6e18547e99f0c75cd3eb9e09d: Bootstrap starting.
I20260619 02:50:21.914094 20596 raft_consensus.cc:729] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 383629b02ca34692b50370b28558e73e, State: Initialized, Role: FOLLOWER
I20260619 02:50:21.915604 20595 tablet_bootstrap.cc:654] T a03a92ee8712475f9430615e968c669d P 0c2b46b6e18547e99f0c75cd3eb9e09d: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:21.914383 20596 consensus_queue.cc:260] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } }
I20260619 02:50:21.916046 20603 tablet_replica.cc:439] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } } }
I20260619 02:50:21.916302 20603 tablet_replica.cc:442] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:50:21.918936 20596 ts_tablet_manager.cc:1428] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20260619 02:50:21.923512 20597 tablet_bootstrap.cc:492] T a03a92ee8712475f9430615e968c669d P 2f694c40a66c412b879c04f7c183cfb5: No bootstrap required, opened a new log
I20260619 02:50:21.923702 20597 ts_tablet_manager.cc:1397] T a03a92ee8712475f9430615e968c669d P 2f694c40a66c412b879c04f7c183cfb5: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.003s
I20260619 02:50:21.924494 20597 raft_consensus.cc:348] T a03a92ee8712475f9430615e968c669d P 2f694c40a66c412b879c04f7c183cfb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } }
I20260619 02:50:21.924752 20597 raft_consensus.cc:374] T a03a92ee8712475f9430615e968c669d P 2f694c40a66c412b879c04f7c183cfb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:21.924829 20597 raft_consensus.cc:729] T a03a92ee8712475f9430615e968c669d P 2f694c40a66c412b879c04f7c183cfb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f694c40a66c412b879c04f7c183cfb5, State: Initialized, Role: FOLLOWER
I20260619 02:50:21.925110 20597 consensus_queue.cc:260] T a03a92ee8712475f9430615e968c669d P 2f694c40a66c412b879c04f7c183cfb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } }
I20260619 02:50:21.925590 20614 tablet_replica.cc:439] T a03a92ee8712475f9430615e968c669d P 2f694c40a66c412b879c04f7c183cfb5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } } }
I20260619 02:50:21.925994 20614 tablet_replica.cc:442] T a03a92ee8712475f9430615e968c669d P 2f694c40a66c412b879c04f7c183cfb5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:50:21.926451 20597 ts_tablet_manager.cc:1428] T a03a92ee8712475f9430615e968c669d P 2f694c40a66c412b879c04f7c183cfb5: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260619 02:50:21.927727 20595 tablet_bootstrap.cc:492] T a03a92ee8712475f9430615e968c669d P 0c2b46b6e18547e99f0c75cd3eb9e09d: No bootstrap required, opened a new log
I20260619 02:50:21.927917 20595 ts_tablet_manager.cc:1397] T a03a92ee8712475f9430615e968c669d P 0c2b46b6e18547e99f0c75cd3eb9e09d: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.006s
I20260619 02:50:21.928658 20595 raft_consensus.cc:348] T a03a92ee8712475f9430615e968c669d P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } }
I20260619 02:50:21.928906 20595 raft_consensus.cc:374] T a03a92ee8712475f9430615e968c669d P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:21.928984 20595 raft_consensus.cc:729] T a03a92ee8712475f9430615e968c669d P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c2b46b6e18547e99f0c75cd3eb9e09d, State: Initialized, Role: FOLLOWER
I20260619 02:50:21.929252 20595 consensus_queue.cc:260] T a03a92ee8712475f9430615e968c669d P 0c2b46b6e18547e99f0c75cd3eb9e09d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } }
I20260619 02:50:21.929834 20595 ts_tablet_manager.cc:1428] T a03a92ee8712475f9430615e968c669d P 0c2b46b6e18547e99f0c75cd3eb9e09d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:50:21.929983 20601 tablet_replica.cc:439] T a03a92ee8712475f9430615e968c669d P 0c2b46b6e18547e99f0c75cd3eb9e09d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } } }
I20260619 02:50:21.930265 20601 tablet_replica.cc:442] T a03a92ee8712475f9430615e968c669d P 0c2b46b6e18547e99f0c75cd3eb9e09d: This TxnStatusTablet replica's current role is: FOLLOWER
W20260619 02:50:21.933034 20299 tablet.cc:2369] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:50:22.005904 20603 raft_consensus.cc:482] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:50:22.006083 20603 raft_consensus.cc:504] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } }
I20260619 02:50:22.007573 20603 leader_election.cc:288] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c2b46b6e18547e99f0c75cd3eb9e09d (127.18.134.1:36831), 2f694c40a66c412b879c04f7c183cfb5 (127.18.134.2:37053)
I20260619 02:50:22.015111 20594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:22.008078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:36155 (local address 127.18.134.2:37053)
0619 02:50:22.008289 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:22.008295 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:22.008827 (+ 532us) server_negotiation.cc:407] Connection header received
0619 02:50:22.009075 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:22.009099 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:22.009169 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:22.009252 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:22.009735 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.010886 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.012778 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.014116 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.014297 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:22.014379 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:22.014672 (+ 293us) server_negotiation.cc:299] Negotiation successful
0619 02:50:22.014827 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 02:50:22.015151 20639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:22.007850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:37053 (local address 127.18.134.3:36155)
0619 02:50:22.008721 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:22.008739 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:22.008939 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:22.009286 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:22.009290 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:22.009314 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:22.009552 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:22.009566 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.011009 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.011015 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:22.012588 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:22.012602 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.014315 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.014321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:22.014489 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:22.014525 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 02:50:22.014623 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 02:50:22.014778 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"thread_start_us":851,"threads_started":1}
I20260619 02:50:22.015838 20638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:22.007899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:36831 (local address 127.18.134.3:33543)
0619 02:50:22.008944 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:22.008960 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:22.009134 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:22.009433 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:22.009437 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:22.009465 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:22.010397 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:22.010413 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.011821 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.011827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:22.013546 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:22.013557 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.015097 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.015103 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:22.015265 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:22.015293 (+ 28us) client_negotiation.cc:769] Sending connection context
0619 02:50:22.015428 (+ 135us) client_negotiation.cc:240] Negotiation successful
0619 02:50:22.015548 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":934,"thread_start_us":515,"threads_started":1}
I20260619 02:50:22.016052 20591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:22.008120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:33543 (local address 127.18.134.1:36831)
0619 02:50:22.008292 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:22.008299 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:22.009039 (+ 740us) server_negotiation.cc:407] Connection header received
0619 02:50:22.009265 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:22.009270 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:22.009336 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:22.009411 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:22.010563 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.011662 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.013692 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.014747 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.014887 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:22.014955 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:22.015685 (+ 730us) server_negotiation.cc:299] Negotiation successful
0619 02:50:22.015812 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 02:50:22.016069 20387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a03a92ee8712475f9430615e968c669d" candidate_uuid: "383629b02ca34692b50370b28558e73e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f694c40a66c412b879c04f7c183cfb5" is_pre_election: true
I20260619 02:50:22.016352 20387 raft_consensus.cc:2454] T a03a92ee8712475f9430615e968c669d P 2f694c40a66c412b879c04f7c183cfb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 383629b02ca34692b50370b28558e73e in term 0.
I20260619 02:50:22.016820 20458 leader_election.cc:302] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f694c40a66c412b879c04f7c183cfb5, 383629b02ca34692b50370b28558e73e; no voters:
I20260619 02:50:22.017055 20248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a03a92ee8712475f9430615e968c669d" candidate_uuid: "383629b02ca34692b50370b28558e73e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" is_pre_election: true
I20260619 02:50:22.017261 20603 raft_consensus.cc:2781] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:50:22.017279 20248 raft_consensus.cc:2454] T a03a92ee8712475f9430615e968c669d P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 383629b02ca34692b50370b28558e73e in term 0.
I20260619 02:50:22.017364 20603 raft_consensus.cc:482] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:50:22.017447 20603 raft_consensus.cc:3037] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:22.020404 20603 raft_consensus.cc:504] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } }
I20260619 02:50:22.021147 20603 leader_election.cc:288] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e [CANDIDATE]: Term 1 election: Requested vote from peers 0c2b46b6e18547e99f0c75cd3eb9e09d (127.18.134.1:36831), 2f694c40a66c412b879c04f7c183cfb5 (127.18.134.2:37053)
I20260619 02:50:22.021538 20248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a03a92ee8712475f9430615e968c669d" candidate_uuid: "383629b02ca34692b50370b28558e73e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d"
I20260619 02:50:22.021755 20248 raft_consensus.cc:3037] T a03a92ee8712475f9430615e968c669d P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:22.024696 20248 raft_consensus.cc:2454] T a03a92ee8712475f9430615e968c669d P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 383629b02ca34692b50370b28558e73e in term 1.
I20260619 02:50:22.025226 20459 leader_election.cc:302] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c2b46b6e18547e99f0c75cd3eb9e09d, 383629b02ca34692b50370b28558e73e; no voters:
I20260619 02:50:22.025609 20603 raft_consensus.cc:2781] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:50:22.029871 20387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a03a92ee8712475f9430615e968c669d" candidate_uuid: "383629b02ca34692b50370b28558e73e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f694c40a66c412b879c04f7c183cfb5"
I20260619 02:50:22.029923 20603 raft_consensus.cc:686] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e [term 1 LEADER]: Becoming Leader. State: Replica: 383629b02ca34692b50370b28558e73e, State: Running, Role: LEADER
I20260619 02:50:22.030511 20387 raft_consensus.cc:3037] T a03a92ee8712475f9430615e968c669d P 2f694c40a66c412b879c04f7c183cfb5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:22.030573 20603 consensus_queue.cc:237] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } }
I20260619 02:50:22.032950 20387 raft_consensus.cc:2454] T a03a92ee8712475f9430615e968c669d P 2f694c40a66c412b879c04f7c183cfb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 383629b02ca34692b50370b28558e73e in term 1.
I20260619 02:50:22.035403 20640 tablet_replica.cc:439] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e: TxnStatusTablet state changed. Reason: New leader 383629b02ca34692b50370b28558e73e. Latest consensus state: current_term: 1 leader_uuid: "383629b02ca34692b50370b28558e73e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } } }
I20260619 02:50:22.035789 20640 tablet_replica.cc:442] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:50:22.036518 20642 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 02:50:22.036727 20642 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 02:50:22.037256 20090 catalog_manager.cc:5495] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e reported cstate change: term changed from 0 to 1, leader changed from <none> to 383629b02ca34692b50370b28558e73e (127.18.134.3). New cstate: current_term: 1 leader_uuid: "383629b02ca34692b50370b28558e73e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } health_report { overall_health: HEALTHY } } }
I20260619 02:50:22.070334 20592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:22.065319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51032 (local address 127.18.134.3:40999)
0619 02:50:22.065491 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:22.065497 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:22.065515 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:50:22.065619 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:22.065623 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:22.065697 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:22.065808 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:50:22.066153 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.066968 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.068186 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.068647 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.068749 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:22.069176 (+ 427us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:50:22.069695 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:50:22.069830 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:22.069931 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 02:50:22.070024 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 02:50:22.070317 20634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:22.065012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:40999 (local address 127.0.0.1:51032)
0619 02:50:22.065229 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:22.065249 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:22.065392 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:22.065849 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:22.065854 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:22.065867 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:50:22.066033 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:22.066041 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.067083 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.067089 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:22.068058 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:22.068069 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.068786 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.068793 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:22.068954 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:22.069008 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:50:22.069797 (+ 789us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:50:22.069826 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 02:50:22.069902 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 02:50:22.070048 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 02:50:22.074023 20248 raft_consensus.cc:1261] T a03a92ee8712475f9430615e968c669d P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 1 FOLLOWER]: Refusing update from remote peer 383629b02ca34692b50370b28558e73e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:50:22.074193 20387 raft_consensus.cc:1261] T a03a92ee8712475f9430615e968c669d P 2f694c40a66c412b879c04f7c183cfb5 [term 1 FOLLOWER]: Refusing update from remote peer 383629b02ca34692b50370b28558e73e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:50:22.074787 20603 consensus_queue.cc:1035] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:50:22.075153 20640 consensus_queue.cc:1035] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:50:22.083626 20601 tablet_replica.cc:439] T a03a92ee8712475f9430615e968c669d P 0c2b46b6e18547e99f0c75cd3eb9e09d: TxnStatusTablet state changed. Reason: New leader 383629b02ca34692b50370b28558e73e. Latest consensus state: current_term: 1 leader_uuid: "383629b02ca34692b50370b28558e73e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } } }
I20260619 02:50:22.084005 20601 tablet_replica.cc:442] T a03a92ee8712475f9430615e968c669d P 0c2b46b6e18547e99f0c75cd3eb9e09d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:50:22.088771 20640 tablet_replica.cc:439] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "383629b02ca34692b50370b28558e73e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } } }
I20260619 02:50:22.089040 20640 tablet_replica.cc:442] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:50:22.089349 20614 tablet_replica.cc:439] T a03a92ee8712475f9430615e968c669d P 2f694c40a66c412b879c04f7c183cfb5: TxnStatusTablet state changed. Reason: New leader 383629b02ca34692b50370b28558e73e. Latest consensus state: current_term: 1 leader_uuid: "383629b02ca34692b50370b28558e73e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } } }
I20260619 02:50:22.089619 20614 tablet_replica.cc:442] T a03a92ee8712475f9430615e968c669d P 2f694c40a66c412b879c04f7c183cfb5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:50:22.090265 20611 mvcc.cc:205] Tried to move back new op lower bound from 7298406080808251392 to 7298406080642424832. Current Snapshot: MvccSnapshot[applied={T|T < 7298406080808251392 or (T in {7298406080808251392})}]
I20260619 02:50:22.098232 20603 tablet_replica.cc:439] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "383629b02ca34692b50370b28558e73e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } } }
I20260619 02:50:22.098539 20603 tablet_replica.cc:442] T a03a92ee8712475f9430615e968c669d P 383629b02ca34692b50370b28558e73e: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:50:22.099781 20609 mvcc.cc:205] Tried to move back new op lower bound from 7298406080808251392 to 7298406080642424832. Current Snapshot: MvccSnapshot[applied={T|T < 7298406080808251392}]
I20260619 02:50:22.100323 20614 tablet_replica.cc:439] T a03a92ee8712475f9430615e968c669d P 2f694c40a66c412b879c04f7c183cfb5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "383629b02ca34692b50370b28558e73e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } } }
I20260619 02:50:22.100539 20614 tablet_replica.cc:442] T a03a92ee8712475f9430615e968c669d P 2f694c40a66c412b879c04f7c183cfb5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:50:22.106189 20601 tablet_replica.cc:439] T a03a92ee8712475f9430615e968c669d P 0c2b46b6e18547e99f0c75cd3eb9e09d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "383629b02ca34692b50370b28558e73e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c2b46b6e18547e99f0c75cd3eb9e09d" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36831 } } peers { permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 } } peers { permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 } } }
I20260619 02:50:22.106494 20601 tablet_replica.cc:442] T a03a92ee8712475f9430615e968c669d P 0c2b46b6e18547e99f0c75cd3eb9e09d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:50:22.107566 20591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:22.097154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39168 (local address 127.18.134.1:36831)
0619 02:50:22.097320 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:22.097326 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:22.097756 (+ 430us) server_negotiation.cc:407] Connection header received
0619 02:50:22.097850 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:22.097855 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:22.097949 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:22.098087 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:50:22.098980 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.100017 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.104366 (+ 4349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.104984 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.105111 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:22.105728 (+ 617us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:50:22.106547 (+ 819us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:50:22.106711 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:22.107208 (+ 497us) server_negotiation.cc:299] Negotiation successful
0619 02:50:22.107286 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 02:50:22.128615 20592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:22.118533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:56703 (local address 127.18.134.3:40999)
0619 02:50:22.118754 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:22.118779 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:22.123279 (+ 4500us) server_negotiation.cc:407] Connection header received
0619 02:50:22.123480 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:22.123484 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:22.123549 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:22.123626 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:50:22.124107 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.124883 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.126796 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.127322 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.127401 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:22.127907 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:50:22.128010 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:50:22.128087 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:22.128214 (+ 127us) server_negotiation.cc:299] Negotiation successful
0619 02:50:22.128295 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260619 02:50:22.128672 20649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:22.118532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:40999 (local address 127.18.134.1:56703)
0619 02:50:22.123156 (+ 4624us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:22.123173 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:22.123349 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:22.123655 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:22.123659 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:22.123671 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:50:22.123968 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:22.123977 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.125006 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.125012 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:22.126597 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:22.126618 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.127555 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.127559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:22.127673 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:22.127727 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:50:22.128102 (+ 375us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:50:22.128113 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 02:50:22.128180 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 02:50:22.128316 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4458,"thread_start_us":4304,"threads_started":1}
I20260619 02:50:22.139750 20387 raft_consensus.cc:1261] T 46259ac9881c42ecb1e60fbe76f13d58 P 2f694c40a66c412b879c04f7c183cfb5 [term 1 FOLLOWER]: Refusing update from remote peer 0c2b46b6e18547e99f0c75cd3eb9e09d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:50:22.139757 20526 raft_consensus.cc:1261] T 46259ac9881c42ecb1e60fbe76f13d58 P 383629b02ca34692b50370b28558e73e [term 1 FOLLOWER]: Refusing update from remote peer 0c2b46b6e18547e99f0c75cd3eb9e09d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:50:22.140560 20627 consensus_queue.cc:1035] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d [LEADER]: Connected to new peer: Peer: permanent_uuid: "383629b02ca34692b50370b28558e73e" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 40999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:50:22.141175 20627 consensus_queue.cc:1035] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f694c40a66c412b879c04f7c183cfb5" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:50:22.215852 20591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:22.198880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:53417 (local address 127.18.134.1:36831)
0619 02:50:22.199043 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:22.199049 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:22.209035 (+ 9986us) server_negotiation.cc:407] Connection header received
0619 02:50:22.209229 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:22.209236 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:22.209333 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:22.209443 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:50:22.210045 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.211262 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.213549 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.214360 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.214490 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:22.215076 (+ 586us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:50:22.215209 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:50:22.215296 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:22.215456 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 02:50:22.215538 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 02:50:22.216090 20655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:22.198428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:36831 (local address 127.18.134.3:53417)
0619 02:50:22.208896 (+ 10468us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:22.208916 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:22.209069 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:22.209461 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:22.209466 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:22.209479 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:50:22.209879 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:22.209891 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.211444 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.211450 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:22.213364 (+ 1914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:22.213382 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.214567 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.214574 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:22.214793 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:22.214884 (+ 91us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:50:22.215316 (+ 432us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:50:22.215328 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 02:50:22.215413 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 02:50:22.215603 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10237,"thread_start_us":10022,"threads_started":1}
I20260619 02:50:22.263023 20092 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:50:22.263294 20092 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39456:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:50:22.266980 20092 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a6c62c7df18c4c549706d19cd35f702a: Sending DeleteTablet for 3 replicas of tablet 46259ac9881c42ecb1e60fbe76f13d58
I20260619 02:50:22.267863 20228 tablet_service.cc:1514] Processing DeleteTablet for tablet 46259ac9881c42ecb1e60fbe76f13d58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:50:22 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:39154
I20260619 02:50:22.268704 20367 tablet_service.cc:1514] Processing DeleteTablet for tablet 46259ac9881c42ecb1e60fbe76f13d58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:50:22 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:40156
I20260619 02:50:22.269492 20506 tablet_service.cc:1514] Processing DeleteTablet for tablet 46259ac9881c42ecb1e60fbe76f13d58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:50:22 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:51030
I20260619 02:50:22.269793 20619 tablet_replica.cc:330] T 46259ac9881c42ecb1e60fbe76f13d58 P 383629b02ca34692b50370b28558e73e: stopping tablet replica
I20260619 02:50:22.270006 20619 raft_consensus.cc:2229] T 46259ac9881c42ecb1e60fbe76f13d58 P 383629b02ca34692b50370b28558e73e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:50:22.270205 20619 raft_consensus.cc:2258] T 46259ac9881c42ecb1e60fbe76f13d58 P 383629b02ca34692b50370b28558e73e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:50:22.271322 20619 ts_tablet_manager.cc:1905] T 46259ac9881c42ecb1e60fbe76f13d58 P 383629b02ca34692b50370b28558e73e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:50:22.278961 20659 tablet_replica.cc:330] T 46259ac9881c42ecb1e60fbe76f13d58 P 2f694c40a66c412b879c04f7c183cfb5: stopping tablet replica
I20260619 02:50:22.279170 20659 raft_consensus.cc:2229] T 46259ac9881c42ecb1e60fbe76f13d58 P 2f694c40a66c412b879c04f7c183cfb5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:50:22.279422 20659 raft_consensus.cc:2258] T 46259ac9881c42ecb1e60fbe76f13d58 P 2f694c40a66c412b879c04f7c183cfb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:50:22.280455 20619 ts_tablet_manager.cc:1918] T 46259ac9881c42ecb1e60fbe76f13d58 P 383629b02ca34692b50370b28558e73e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:50:22.280498 20659 ts_tablet_manager.cc:1905] T 46259ac9881c42ecb1e60fbe76f13d58 P 2f694c40a66c412b879c04f7c183cfb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:50:22.280619 20619 log.cc:1196] T 46259ac9881c42ecb1e60fbe76f13d58 P 383629b02ca34692b50370b28558e73e: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-2/wal/wals/46259ac9881c42ecb1e60fbe76f13d58
I20260619 02:50:22.281092 20619 ts_tablet_manager.cc:1939] T 46259ac9881c42ecb1e60fbe76f13d58 P 383629b02ca34692b50370b28558e73e: Deleting consensus metadata
I20260619 02:50:22.282178 20076 catalog_manager.cc:4841] TS 383629b02ca34692b50370b28558e73e (127.18.134.3:40999): tablet 46259ac9881c42ecb1e60fbe76f13d58 (table test-table [id=8e37c48a3b4d4362852bcf1104c5d6c3]) successfully deleted
I20260619 02:50:22.287770 20659 ts_tablet_manager.cc:1918] T 46259ac9881c42ecb1e60fbe76f13d58 P 2f694c40a66c412b879c04f7c183cfb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:50:22.287922 20659 log.cc:1196] T 46259ac9881c42ecb1e60fbe76f13d58 P 2f694c40a66c412b879c04f7c183cfb5: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-1/wal/wals/46259ac9881c42ecb1e60fbe76f13d58
I20260619 02:50:22.288353 20659 ts_tablet_manager.cc:1939] T 46259ac9881c42ecb1e60fbe76f13d58 P 2f694c40a66c412b879c04f7c183cfb5: Deleting consensus metadata
I20260619 02:50:22.289523 20077 catalog_manager.cc:4841] TS 2f694c40a66c412b879c04f7c183cfb5 (127.18.134.2:37053): tablet 46259ac9881c42ecb1e60fbe76f13d58 (table test-table [id=8e37c48a3b4d4362852bcf1104c5d6c3]) successfully deleted
I20260619 02:50:22.295203 20658 tablet_replica.cc:330] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d: stopping tablet replica
I20260619 02:50:22.295682 20658 raft_consensus.cc:2229] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:50:22.296681 20658 raft_consensus.cc:2258] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:50:22.297971 20658 ts_tablet_manager.cc:1905] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:50:22.305554 20658 ts_tablet_manager.cc:1918] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:50:22.306430 20658 log.cc:1196] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781837410515220-18968-0/minicluster-data/ts-0/wal/wals/46259ac9881c42ecb1e60fbe76f13d58
I20260619 02:50:22.306968 20658 ts_tablet_manager.cc:1939] T 46259ac9881c42ecb1e60fbe76f13d58 P 0c2b46b6e18547e99f0c75cd3eb9e09d: Deleting consensus metadata
I20260619 02:50:22.308174 20078 catalog_manager.cc:4841] TS 0c2b46b6e18547e99f0c75cd3eb9e09d (127.18.134.1:36831): tablet 46259ac9881c42ecb1e60fbe76f13d58 (table test-table [id=8e37c48a3b4d4362852bcf1104c5d6c3]) successfully deleted
Jun 19 02:50:22 dist-test-slave-9mr0 krb5kdc[20047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837421, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.134.1@KRBTEST.COM
I20260619 02:50:22.325932 20591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:22.293686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39176 (local address 127.18.134.1:36831)
0619 02:50:22.293883 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:22.293890 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:22.306382 (+ 12492us) server_negotiation.cc:407] Connection header received
0619 02:50:22.306765 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:22.306774 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:22.306883 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:22.306969 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:22.309653 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.310879 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.312724 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.313378 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.313506 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:22.318609 (+ 5103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:22.318649 (+ 40us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:22.318671 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:22.318727 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:22.322766 (+ 4039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:22.323593 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:22.323602 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:22.323606 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:22.323784 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:22.324178 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:22.324182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:22.324185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:22.324637 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:22.325143 (+ 506us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:22.325299 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 02:50:22.325540 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260619 02:50:22.326659 20293 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39176
I20260619 02:50:22.346612 20140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:22.330304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39472 (local address 127.18.134.62:39567)
0619 02:50:22.330529 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:22.330535 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:22.331099 (+ 564us) server_negotiation.cc:407] Connection header received
0619 02:50:22.331254 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:22.331260 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:22.331339 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:22.331445 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:22.333137 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.334219 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.336196 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.336893 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.337027 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:22.339744 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:22.339751 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:22.339756 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:22.339799 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:22.343214 (+ 3415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:22.343991 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:22.343997 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:22.344000 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:22.344153 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:22.344532 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:22.344537 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:22.344540 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:22.344889 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:22.345142 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:22.345472 (+ 330us) server_negotiation.cc:299] Negotiation successful
0619 02:50:22.346198 (+ 726us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260619 02:50:22.347189 20092 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39472
I20260619 02:50:22.350040 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 20163
I20260619 02:50:22.370481 20591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:22.351593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39184 (local address 127.18.134.1:36831)
0619 02:50:22.351777 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:22.351783 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:22.355136 (+ 3353us) server_negotiation.cc:407] Connection header received
0619 02:50:22.355286 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:22.355358 (+ 72us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:22.355452 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:22.355565 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:22.357101 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.358357 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.360141 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.360854 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.360979 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:22.363799 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:22.363806 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:22.363820 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:22.363870 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:22.367409 (+ 3539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:22.368204 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:22.368210 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:22.368213 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:22.368383 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:22.368752 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:22.368757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:22.368760 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:22.369088 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:22.369339 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:22.369648 (+ 309us) server_negotiation.cc:299] Negotiation successful
0619 02:50:22.369888 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260619 02:50:22.371127 20293 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39184
W20260619 02:50:22.371747 18968 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 20163: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 02:50:22.371954 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 20163
I20260619 02:50:22.386921 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 20302
Jun 19 02:50:22 dist-test-slave-9mr0 krb5kdc[20047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837421, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.134.2@KRBTEST.COM
I20260619 02:50:22.403179 20594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:22.387651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40178 (local address 127.18.134.2:37053)
0619 02:50:22.387935 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:22.387942 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:22.388470 (+ 528us) server_negotiation.cc:407] Connection header received
0619 02:50:22.388640 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:22.388646 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:22.388716 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:22.388796 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:22.389877 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.390540 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.391884 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.392274 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.392380 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:22.396340 (+ 3960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:22.396358 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:22.396375 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:22.396425 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:22.400186 (+ 3761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:22.400978 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:22.400987 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:22.400991 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:22.401155 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:22.401569 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:22.401574 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:22.401577 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:22.402002 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:22.402315 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:22.402575 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 02:50:22.402820 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260619 02:50:22.403756 20432 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40178
W20260619 02:50:22.404361 18968 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 20302: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 02:50:22.404522 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 20302
I20260619 02:50:22.430794 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 20441
Jun 19 02:50:22 dist-test-slave-9mr0 krb5kdc[20047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837421, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.134.3@KRBTEST.COM
I20260619 02:50:22.449538 20592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:22.431904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51046 (local address 127.18.134.3:40999)
0619 02:50:22.432078 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:22.432084 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:22.432104 (+ 20us) server_negotiation.cc:407] Connection header received
0619 02:50:22.432160 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:22.432165 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:22.432254 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:22.432368 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:22.433927 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.435126 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.436900 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:22.437552 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:22.437689 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:22.442491 (+ 4802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:22.442507 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:22.442527 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:22.442581 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:22.446507 (+ 3926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:22.447463 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:22.447477 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:22.447480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:22.447650 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:22.447995 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:22.448001 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:22.448004 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:22.448447 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:22.448786 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:22.448938 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 02:50:22.449157 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260619 02:50:22.450217 20570 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51046
W20260619 02:50:22.450884 18968 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 20441: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 02:50:22.451006 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 20441
I20260619 02:50:22.466550 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 20060
I20260619 02:50:22.467633 20132 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:22.577129 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 20060
2026-06-19T02:50:22Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4289 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:50:22 dist-test-slave-9mr0 krb5kdc[20668](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:50:22 dist-test-slave-9mr0 krb5kdc[20668](info): set up 2 sockets
Jun 19 02:50:22 dist-test-slave-9mr0 krb5kdc[20668](info): commencing operation
krb5kdc: starting...
W20260619 02:50:23.578665 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.963s user 0.002s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:50:23 dist-test-slave-9mr0 krb5kdc[20668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:50:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:50:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:50:23.762522 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:35529
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:46617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:35529
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:24.069026 20681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:24.069324 20681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:24.069423 20681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:24.076527 20681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:50:24.076676 20681 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 02:50:24.076740 20681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:24.076802 20681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:24.076850 20681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:50:24.076908 20681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:50:24.086704 20681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:46617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:35529
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:35529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:24.087953 20681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:24.089193 20681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:24.100270 20689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:24.100497 20687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:24.100840 20686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:24.101222 20681 server_base.cc:1034] running on GCE node
I20260619 02:50:24.102061 20681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:24.103552 20681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:24.104780 20681 hybrid_clock.cc:648] HybridClock initialized: now 1781837424104710 us; error 53 us; skew 500 ppm
I20260619 02:50:24.105129 20681 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:24 dist-test-slave-9mr0 krb5kdc[20668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837424, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:24.108177 20681 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:50:24.109611 20681 webserver.cc:460] Webserver started at http://127.18.134.62:44275/ using document root <none> and password file <none>
I20260619 02:50:24.110281 20681 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:24.110365 20681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:24.110612 20681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:24.113493 20681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "f6a59ebf29ad4eeab57577c61122f952"
format_stamp: "Formatted at 2026-06-19 02:50:24 on dist-test-slave-9mr0"
server_key: "56dbe8f2c632bd26aba318cf1c3e687e"
server_key_iv: "83e1d2f7c24663f158a51af214bc0890"
server_key_version: "encryptionkey@0"
I20260619 02:50:24.114204 20681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "f6a59ebf29ad4eeab57577c61122f952"
format_stamp: "Formatted at 2026-06-19 02:50:24 on dist-test-slave-9mr0"
server_key: "56dbe8f2c632bd26aba318cf1c3e687e"
server_key_iv: "83e1d2f7c24663f158a51af214bc0890"
server_key_version: "encryptionkey@0"
I20260619 02:50:24.119342 20681 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 02:50:24.122748 20696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:24.124084 20681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:50:24.124253 20681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "f6a59ebf29ad4eeab57577c61122f952"
format_stamp: "Formatted at 2026-06-19 02:50:24 on dist-test-slave-9mr0"
server_key: "56dbe8f2c632bd26aba318cf1c3e687e"
server_key_iv: "83e1d2f7c24663f158a51af214bc0890"
server_key_version: "encryptionkey@0"
I20260619 02:50:24.124457 20681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:24.139050 20681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:24.143101 20681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:24.143342 20681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:24.163064 20681 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:35529
I20260619 02:50:24.163081 20757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:35529 every 8 connection(s)
I20260619 02:50:24.164420 20681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:50:24.168183 20758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:24.168706 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 20681
I20260619 02:50:24.168896 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:50:24.169680 18968 external_mini_cluster.cc:1428] Setting key 7cf1c2d8ec18970c818932e536144254
I20260619 02:50:24.180703 20758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6a59ebf29ad4eeab57577c61122f952: Bootstrap starting.
Jun 19 02:50:24 dist-test-slave-9mr0 krb5kdc[20668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837423, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:24.184047 20758 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6a59ebf29ad4eeab57577c61122f952: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:24.185132 20758 log.cc:824] T 00000000000000000000000000000000 P f6a59ebf29ad4eeab57577c61122f952: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:24.188030 20758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6a59ebf29ad4eeab57577c61122f952: No bootstrap required, opened a new log
I20260619 02:50:24.191392 20761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:24.171765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58742 (local address 127.18.134.62:35529)
0619 02:50:24.172539 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:24.172551 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:24.172589 (+ 38us) server_negotiation.cc:407] Connection header received
0619 02:50:24.174539 (+ 1950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:24.174567 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:24.175092 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:24.175498 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:24.176929 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:24.178335 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:24.179595 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:24.180283 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:24.180459 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:24.184432 (+ 3973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:24.184440 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:24.184456 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:24.184502 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:24.187525 (+ 3023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:24.188155 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:24.188163 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:24.188171 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:24.188345 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:24.188690 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:24.188694 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:24.188696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:24.189209 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:24.189492 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:24.189928 (+ 436us) server_negotiation.cc:299] Negotiation successful
0619 02:50:24.190330 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":345,"threads_started":1}
I20260619 02:50:24.195627 20758 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6a59ebf29ad4eeab57577c61122f952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a59ebf29ad4eeab57577c61122f952" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35529 } }
I20260619 02:50:24.195981 20758 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6a59ebf29ad4eeab57577c61122f952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:24.196071 20758 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6a59ebf29ad4eeab57577c61122f952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6a59ebf29ad4eeab57577c61122f952, State: Initialized, Role: FOLLOWER
I20260619 02:50:24.196635 20758 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6a59ebf29ad4eeab57577c61122f952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a59ebf29ad4eeab57577c61122f952" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35529 } }
I20260619 02:50:24.196856 20758 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6a59ebf29ad4eeab57577c61122f952 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:50:24.196982 20758 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6a59ebf29ad4eeab57577c61122f952 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:50:24.197134 20758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6a59ebf29ad4eeab57577c61122f952 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:24.199322 20758 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6a59ebf29ad4eeab57577c61122f952 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a59ebf29ad4eeab57577c61122f952" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35529 } }
I20260619 02:50:24.199903 20758 leader_election.cc:302] T 00000000000000000000000000000000 P f6a59ebf29ad4eeab57577c61122f952 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6a59ebf29ad4eeab57577c61122f952; no voters:
I20260619 02:50:24.200559 20758 leader_election.cc:288] T 00000000000000000000000000000000 P f6a59ebf29ad4eeab57577c61122f952 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:50:24.200654 20763 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6a59ebf29ad4eeab57577c61122f952 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:50:24.201296 20763 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6a59ebf29ad4eeab57577c61122f952 [term 1 LEADER]: Becoming Leader. State: Replica: f6a59ebf29ad4eeab57577c61122f952, State: Running, Role: LEADER
I20260619 02:50:24.201797 20763 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6a59ebf29ad4eeab57577c61122f952 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a59ebf29ad4eeab57577c61122f952" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35529 } }
I20260619 02:50:24.202497 20758 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6a59ebf29ad4eeab57577c61122f952 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:50:24.204450 20764 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6a59ebf29ad4eeab57577c61122f952 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6a59ebf29ad4eeab57577c61122f952" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a59ebf29ad4eeab57577c61122f952" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35529 } } }
I20260619 02:50:24.204406 20765 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6a59ebf29ad4eeab57577c61122f952 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6a59ebf29ad4eeab57577c61122f952. Latest consensus state: current_term: 1 leader_uuid: "f6a59ebf29ad4eeab57577c61122f952" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a59ebf29ad4eeab57577c61122f952" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35529 } } }
I20260619 02:50:24.204860 20765 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6a59ebf29ad4eeab57577c61122f952 [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:24.205430 20764 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6a59ebf29ad4eeab57577c61122f952 [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:24.205847 20770 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:50:24.211494 20770 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:50:24.217085 20780 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 02:50:24.221352 20770 catalog_manager.cc:1347] Generated new cluster ID: 7051a9621ceb49f58f861c64ac1dba53
I20260619 02:50:24.221472 20770 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:50:24.248243 20770 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:50:24.249624 20770 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:50:24.263962 20770 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6a59ebf29ad4eeab57577c61122f952: Generated new TSK 0
I20260619 02:50:24.264822 20770 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.134.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
I20260619 02:50:24.311805 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:0
--local_ip_for_outbound_sockets=127.18.134.1
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:35529
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:46617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:24.622498 20784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:24.622798 20784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:24.622944 20784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:24.631260 20784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:24.631392 20784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:24.631528 20784 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 02:50:24.631593 20784 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:50:24.631696 20784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:50:24.641693 20784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:46617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.134.62:35529
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:24.643021 20784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:24.644336 20784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:24.656703 20792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:24.656741 20789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:24.657181 20790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:24.657186 20784 server_base.cc:1034] running on GCE node
I20260619 02:50:24.657833 20784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:24.658824 20784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:24.660034 20784 hybrid_clock.cc:648] HybridClock initialized: now 1781837424659985 us; error 43 us; skew 500 ppm
I20260619 02:50:24.660329 20784 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:24 dist-test-slave-9mr0 krb5kdc[20668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837424, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:24.663233 20784 init.cc:377] Logged in from keytab as kudu/127.18.134.1@KRBTEST.COM (short username kudu)
I20260619 02:50:24.664552 20784 webserver.cc:460] Webserver started at http://127.18.134.1:35581/ using document root <none> and password file <none>
I20260619 02:50:24.665212 20784 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:24.665323 20784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:24.665606 20784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:24.668483 20784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-0/data/instance:
uuid: "e1412605cc804b8eb1a5a66a30889725"
format_stamp: "Formatted at 2026-06-19 02:50:24 on dist-test-slave-9mr0"
server_key: "bad0b06cd970d3f616c129b07fbaaa76"
server_key_iv: "367ba96a5a6707c02efc4345ae07d59b"
server_key_version: "encryptionkey@0"
I20260619 02:50:24.669202 20784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance:
uuid: "e1412605cc804b8eb1a5a66a30889725"
format_stamp: "Formatted at 2026-06-19 02:50:24 on dist-test-slave-9mr0"
server_key: "bad0b06cd970d3f616c129b07fbaaa76"
server_key_iv: "367ba96a5a6707c02efc4345ae07d59b"
server_key_version: "encryptionkey@0"
I20260619 02:50:24.674296 20784 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 02:50:24.677814 20799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:24.679342 20784 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:50:24.679504 20784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "e1412605cc804b8eb1a5a66a30889725"
format_stamp: "Formatted at 2026-06-19 02:50:24 on dist-test-slave-9mr0"
server_key: "bad0b06cd970d3f616c129b07fbaaa76"
server_key_iv: "367ba96a5a6707c02efc4345ae07d59b"
server_key_version: "encryptionkey@0"
I20260619 02:50:24.679697 20784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:24.703433 20784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:24.707892 20784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:24.708179 20784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:24.710786 20784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:24.710896 20784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:24.711024 20784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:24.711107 20784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:50:24 dist-test-slave-9mr0 krb5kdc[20668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837424, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:24.737792 20809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:24.712004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35529 (local address 127.18.134.1:45367)
0619 02:50:24.712960 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:24.713016 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:24.715568 (+ 2552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:24.718787 (+ 3219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:24.718799 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:24.719679 (+ 880us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:24.720720 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:24.720761 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:24.721982 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:24.721991 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:24.724547 (+ 2556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:24.724562 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:24.725520 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:24.725528 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:24.725833 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:24.726591 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:24.726624 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:24.729994 (+ 3370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:24.733422 (+ 3428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:24.733432 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:24.733449 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:24.733838 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:24.734182 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:24.734187 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:24.734191 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:24.734391 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:24.734948 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:24.734956 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:24.735434 (+ 478us) client_negotiation.cc:769] Sending connection context
0619 02:50:24.735731 (+ 297us) client_negotiation.cc:240] Negotiation successful
0619 02:50:24.736290 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":364,"threads_started":1}
I20260619 02:50:24.739101 20808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:24.711197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:45367 (local address 127.18.134.62:35529)
0619 02:50:24.718105 (+ 6908us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:24.718109 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:24.718126 (+ 17us) server_negotiation.cc:407] Connection header received
0619 02:50:24.718180 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:24.718184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:24.718244 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:24.718343 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:24.720959 (+ 2616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:24.721824 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:24.724711 (+ 2887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:24.725247 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:24.725387 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:24.730313 (+ 4926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:24.730318 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:24.730321 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:24.730354 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:24.733273 (+ 2919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:24.733963 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:24.733969 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:24.733972 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:24.734084 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:24.734502 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:24.734507 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:24.734511 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:24.734824 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:24.734961 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:24.738550 (+ 3589us) server_negotiation.cc:299] Negotiation successful
0619 02:50:24.738806 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6807,"thread_start_us":6720,"threads_started":1}
I20260619 02:50:24.757586 20808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:24.744415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:33371 (local address 127.18.134.62:35529)
0619 02:50:24.744607 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:24.744613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:24.745138 (+ 525us) server_negotiation.cc:407] Connection header received
0619 02:50:24.745335 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:24.745339 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:24.745404 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:24.745481 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:24.746685 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:24.747389 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:24.749416 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:24.749821 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:24.749923 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:24.752378 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:24.752384 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:24.752388 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:24.752418 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:24.755007 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:24.755931 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:24.755936 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:24.755940 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:24.756073 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:24.756433 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:24.756436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:24.756438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:24.756688 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:24.756799 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:24.757196 (+ 397us) server_negotiation.cc:299] Negotiation successful
0619 02:50:24.757359 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:50:24.757969 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:24.744403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35529 (local address 127.18.134.1:33371)
0619 02:50:24.745023 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:24.745044 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:24.745210 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:24.745528 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:24.745532 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:24.746131 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:24.746525 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:24.746538 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:24.747564 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:24.747571 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:24.749264 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:24.749281 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:24.749988 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:24.749997 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:24.750178 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:24.750944 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:24.750971 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:24.752200 (+ 1229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:24.755137 (+ 2937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:24.755146 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:24.755150 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:24.755811 (+ 661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:24.756176 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:24.756182 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:24.756186 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:24.756339 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:24.756783 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:24.756788 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:24.756986 (+ 198us) client_negotiation.cc:769] Sending connection context
0619 02:50:24.757131 (+ 145us) client_negotiation.cc:240] Negotiation successful
0619 02:50:24.757470 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":312,"threads_started":1}
I20260619 02:50:24.762696 20784 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:38701
I20260619 02:50:24.762748 20919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:38701 every 8 connection(s)
I20260619 02:50:24.763892 20784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:50:24.765861 20920 heartbeater.cc:347] Connected to a master server at 127.18.134.62:35529
I20260619 02:50:24.766083 20920 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:24.766757 20920 heartbeater.cc:511] Master 127.18.134.62:35529 requested a full tablet report, sending...
I20260619 02:50:24.768679 20713 ts_manager.cc:194] Registered new tserver with Master: e1412605cc804b8eb1a5a66a30889725 (127.18.134.1:38701)
I20260619 02:50:24.769608 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 20784
I20260619 02:50:24.769804 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance
I20260619 02:50:24.770529 18968 external_mini_cluster.cc:1428] Setting key 90fa9a46f35af9dc3ceb039a5590805c
I20260619 02:50:24.770570 20713 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.1@KRBTEST.COM'} at 127.18.134.1:45367
WARNING: no policy specified for kudu/127.18.134.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
I20260619 02:50:24.806033 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:0
--local_ip_for_outbound_sockets=127.18.134.2
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:35529
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:46617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:25.117357 20924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:25.117638 20924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:25.117750 20924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:25.125329 20924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:25.125454 20924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:25.125552 20924 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 02:50:25.125603 20924 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:50:25.125671 20924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:50:25.135525 20924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:46617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.134.62:35529
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:25.136732 20924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:25.137971 20924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:25.150614 20930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:25.150691 20932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:25.150743 20929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:25.151336 20924 server_base.cc:1034] running on GCE node
I20260619 02:50:25.151914 20924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:25.152793 20924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:25.153982 20924 hybrid_clock.cc:648] HybridClock initialized: now 1781837425153931 us; error 43 us; skew 500 ppm
I20260619 02:50:25.154265 20924 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:25 dist-test-slave-9mr0 krb5kdc[20668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837425, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:25.157239 20924 init.cc:377] Logged in from keytab as kudu/127.18.134.2@KRBTEST.COM (short username kudu)
I20260619 02:50:25.158586 20924 webserver.cc:460] Webserver started at http://127.18.134.2:34831/ using document root <none> and password file <none>
I20260619 02:50:25.159252 20924 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:25.159373 20924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:25.159648 20924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:25.162510 20924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-1/data/instance:
uuid: "e2beb7aca7cf48289f69387c69077692"
format_stamp: "Formatted at 2026-06-19 02:50:25 on dist-test-slave-9mr0"
server_key: "acf8ad7c7682aac05ec2a931b884f26c"
server_key_iv: "e2bf651c017edde59c2d536ac516a22a"
server_key_version: "encryptionkey@0"
I20260619 02:50:25.163475 20924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance:
uuid: "e2beb7aca7cf48289f69387c69077692"
format_stamp: "Formatted at 2026-06-19 02:50:25 on dist-test-slave-9mr0"
server_key: "acf8ad7c7682aac05ec2a931b884f26c"
server_key_iv: "e2bf651c017edde59c2d536ac516a22a"
server_key_version: "encryptionkey@0"
I20260619 02:50:25.169682 20924 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260619 02:50:25.173870 20939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:25.175494 20924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:50:25.175639 20924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "e2beb7aca7cf48289f69387c69077692"
format_stamp: "Formatted at 2026-06-19 02:50:25 on dist-test-slave-9mr0"
server_key: "acf8ad7c7682aac05ec2a931b884f26c"
server_key_iv: "e2bf651c017edde59c2d536ac516a22a"
server_key_version: "encryptionkey@0"
I20260619 02:50:25.175845 20924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:25.222303 20924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:25.226423 20924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:25.226715 20924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:25.229359 20924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:25.229465 20924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:25.229602 20924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:25.229674 20924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:50:25 dist-test-slave-9mr0 krb5kdc[20668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837425, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:25.254565 20948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:25.230699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35529 (local address 127.18.134.2:48155)
0619 02:50:25.231541 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:25.231594 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:25.234201 (+ 2607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:25.235174 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:25.235190 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:25.235984 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:25.236761 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:25.236782 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.238034 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.238040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:25.240552 (+ 2512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:25.240572 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.241702 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.241709 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:25.241925 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:25.242570 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:25.242592 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:25.245618 (+ 3026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:25.249409 (+ 3791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:25.249418 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:25.249440 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:25.249885 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:25.250248 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:25.250254 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:25.250258 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:25.250449 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:25.251061 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:25.251072 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:25.251613 (+ 541us) client_negotiation.cc:769] Sending connection context
0619 02:50:25.251994 (+ 381us) client_negotiation.cc:240] Negotiation successful
0619 02:50:25.252648 (+ 654us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":326,"threads_started":1}
I20260619 02:50:25.255228 20808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:25.230072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:48155 (local address 127.18.134.62:35529)
0619 02:50:25.230239 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:25.230246 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:25.231725 (+ 1479us) server_negotiation.cc:407] Connection header received
0619 02:50:25.234583 (+ 2858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:25.234590 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:25.234740 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:25.234850 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:25.236941 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.237868 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.240768 (+ 2900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.241463 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.241593 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:25.245874 (+ 4281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:25.245889 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:25.245894 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:25.245936 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:25.249266 (+ 3330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:25.250029 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:25.250034 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:25.250038 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:25.250149 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:25.250563 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:25.250568 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:25.250572 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:25.250906 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:25.251060 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:25.254603 (+ 3543us) server_negotiation.cc:299] Negotiation successful
0619 02:50:25.254894 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 02:50:25.276664 20808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:25.261234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:36595 (local address 127.18.134.62:35529)
0619 02:50:25.261390 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:25.261395 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:25.264568 (+ 3173us) server_negotiation.cc:407] Connection header received
0619 02:50:25.264694 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:25.264697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:25.264767 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:25.264860 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:25.265840 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.266543 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.268074 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.268621 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.268734 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:25.270965 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:25.270970 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:25.270974 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:25.271005 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:25.274023 (+ 3018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:25.274739 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:25.274745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:25.274747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:25.274882 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:25.275229 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:25.275232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:25.275234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:25.275562 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:25.275689 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:25.276096 (+ 407us) server_negotiation.cc:299] Negotiation successful
0619 02:50:25.276302 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 02:50:25.276758 21033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:25.260776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35529 (local address 127.18.134.2:36595)
0619 02:50:25.264466 (+ 3690us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:25.264486 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:25.264609 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:25.264904 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:25.264909 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:25.265405 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:25.265705 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:25.265715 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.266695 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.266699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:25.267941 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:25.267953 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.268811 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.268821 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:25.268979 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:25.269706 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:25.269731 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:25.270759 (+ 1028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:25.274157 (+ 3398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:25.274163 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:25.274168 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:25.274578 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:25.274989 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:25.274994 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:25.274998 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:25.275123 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:25.275672 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:25.275677 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:25.275853 (+ 176us) client_negotiation.cc:769] Sending connection context
0619 02:50:25.275975 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 02:50:25.276304 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3543,"thread_start_us":3416,"threads_started":1}
I20260619 02:50:25.278425 20924 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:42985
I20260619 02:50:25.278442 21058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:42985 every 8 connection(s)
I20260619 02:50:25.279873 20924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:50:25.281828 21059 heartbeater.cc:347] Connected to a master server at 127.18.134.62:35529
I20260619 02:50:25.282058 21059 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:25.282277 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 20924
I20260619 02:50:25.282444 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance
I20260619 02:50:25.283046 18968 external_mini_cluster.cc:1428] Setting key 86d287565ca880ea74e8831b92aed846
I20260619 02:50:25.282985 21059 heartbeater.cc:511] Master 127.18.134.62:35529 requested a full tablet report, sending...
I20260619 02:50:25.284794 20712 ts_manager.cc:194] Registered new tserver with Master: e2beb7aca7cf48289f69387c69077692 (127.18.134.2:42985)
I20260619 02:50:25.285656 20712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.2@KRBTEST.COM'} at 127.18.134.2:48155
WARNING: no policy specified for kudu/127.18.134.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
I20260619 02:50:25.317544 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:0
--local_ip_for_outbound_sockets=127.18.134.3
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:35529
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:46617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:25.620193 21063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:25.620460 21063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:25.620589 21063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:25.627676 21063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:25.627791 21063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:25.627910 21063 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 02:50:25.627966 21063 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:50:25.628094 21063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:50:25.638566 21063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:46617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.134.62:35529
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:25.639844 21063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:25.641222 21063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:25.654403 21069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:25.654379 21068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:25.654858 21071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:25.655242 21063 server_base.cc:1034] running on GCE node
I20260619 02:50:25.655812 21063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:25.656708 21063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:25.657903 21063 hybrid_clock.cc:648] HybridClock initialized: now 1781837425657848 us; error 49 us; skew 500 ppm
I20260619 02:50:25.658236 21063 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:25 dist-test-slave-9mr0 krb5kdc[20668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837425, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:25.661198 21063 init.cc:377] Logged in from keytab as kudu/127.18.134.3@KRBTEST.COM (short username kudu)
I20260619 02:50:25.662581 21063 webserver.cc:460] Webserver started at http://127.18.134.3:39759/ using document root <none> and password file <none>
I20260619 02:50:25.663260 21063 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:25.663372 21063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:25.663666 21063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:25.666477 21063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-2/data/instance:
uuid: "9046f6d8825d48ee9f249557a32d1e38"
format_stamp: "Formatted at 2026-06-19 02:50:25 on dist-test-slave-9mr0"
server_key: "d132ef3173552ff74557b8aea0e8d841"
server_key_iv: "d585e3d54c2d7f83ba807bbf52277a8d"
server_key_version: "encryptionkey@0"
I20260619 02:50:25.667227 21063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance:
uuid: "9046f6d8825d48ee9f249557a32d1e38"
format_stamp: "Formatted at 2026-06-19 02:50:25 on dist-test-slave-9mr0"
server_key: "d132ef3173552ff74557b8aea0e8d841"
server_key_iv: "d585e3d54c2d7f83ba807bbf52277a8d"
server_key_version: "encryptionkey@0"
I20260619 02:50:25.672272 21063 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 02:50:25.675619 21078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:25.676997 21063 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 02:50:25.677146 21063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "9046f6d8825d48ee9f249557a32d1e38"
format_stamp: "Formatted at 2026-06-19 02:50:25 on dist-test-slave-9mr0"
server_key: "d132ef3173552ff74557b8aea0e8d841"
server_key_iv: "d585e3d54c2d7f83ba807bbf52277a8d"
server_key_version: "encryptionkey@0"
I20260619 02:50:25.677323 21063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:25.689042 21063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:25.693091 21063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:25.693361 21063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:25.696043 21063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:25.696146 21063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:25.696254 21063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:25.696327 21063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:50:25 dist-test-slave-9mr0 krb5kdc[20668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837425, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:25.721189 20808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:25.696760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:34309 (local address 127.18.134.62:35529)
0619 02:50:25.696961 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:25.696969 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:25.698372 (+ 1403us) server_negotiation.cc:407] Connection header received
0619 02:50:25.700739 (+ 2367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:25.700746 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:25.700836 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:25.700934 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:25.703375 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.704396 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.707109 (+ 2713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.707727 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.707864 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:25.713160 (+ 5296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:25.713168 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:25.713173 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:25.713218 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:25.716678 (+ 3460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:25.717796 (+ 1118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:25.717802 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:25.717807 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:25.717969 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:25.718693 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:25.718699 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:25.718702 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:25.719118 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:25.719263 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:25.720538 (+ 1275us) server_negotiation.cc:299] Negotiation successful
0619 02:50:25.720792 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 02:50:25.724030 21087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:25.697334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35529 (local address 127.18.134.3:34309)
0619 02:50:25.698193 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:25.698251 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:25.700380 (+ 2129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:25.701239 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:25.701249 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:25.702090 (+ 841us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:25.703181 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:25.703209 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.704614 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.704621 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:25.706866 (+ 2245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:25.706881 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.707984 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.707993 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:25.708415 (+ 422us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:25.709314 (+ 899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:25.709354 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:25.712835 (+ 3481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:25.716880 (+ 4045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:25.716895 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:25.716923 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:25.717633 (+ 710us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:25.718139 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:25.718146 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:25.718150 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:25.718492 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:25.719354 (+ 862us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:25.719363 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:25.719983 (+ 620us) client_negotiation.cc:769] Sending connection context
0619 02:50:25.721493 (+ 1510us) client_negotiation.cc:240] Negotiation successful
0619 02:50:25.722326 (+ 833us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":355,"threads_started":1}
I20260619 02:50:25.737679 21063 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:42225
I20260619 02:50:25.737716 21197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:42225 every 8 connection(s)
I20260619 02:50:25.738991 21063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:50:25.741354 21198 heartbeater.cc:347] Connected to a master server at 127.18.134.62:35529
I20260619 02:50:25.741595 21198 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:25.742322 21198 heartbeater.cc:511] Master 127.18.134.62:35529 requested a full tablet report, sending...
I20260619 02:50:25.742522 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 21063
I20260619 02:50:25.742699 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance
I20260619 02:50:25.743214 18968 external_mini_cluster.cc:1428] Setting key fb18c51b597f05dd6f7d92848ac2f26b
I20260619 02:50:25.743781 20808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:25.730444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:36095 (local address 127.18.134.62:35529)
0619 02:50:25.730674 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:25.730681 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:25.731252 (+ 571us) server_negotiation.cc:407] Connection header received
0619 02:50:25.731409 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:25.731415 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:25.731502 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:25.731615 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:25.732496 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.733657 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.735397 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.735935 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.736050 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:25.737891 (+ 1841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:25.737898 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:25.737903 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:25.737939 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:25.740973 (+ 3034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:25.741727 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:25.741733 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:25.741736 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:25.741864 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:25.742234 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:25.742240 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:25.742243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:25.742602 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:25.742787 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:25.743219 (+ 432us) server_negotiation.cc:299] Negotiation successful
0619 02:50:25.743465 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 02:50:25.743814 21187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:25.730476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35529 (local address 127.18.134.3:36095)
0619 02:50:25.731140 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:25.731170 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:25.731299 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:25.731621 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:25.731623 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:25.732052 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:25.732360 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:25.732370 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.733814 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.733818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:25.735241 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:25.735264 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.736110 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.736117 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:25.736268 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:25.736874 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:25.736898 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:25.737707 (+ 809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:25.741096 (+ 3389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:25.741100 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:25.741102 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:25.741624 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:25.741971 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:25.741976 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:25.741980 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:25.742103 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:25.742743 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:25.742748 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:25.742949 (+ 201us) client_negotiation.cc:769] Sending connection context
0619 02:50:25.743101 (+ 152us) client_negotiation.cc:240] Negotiation successful
0619 02:50:25.743407 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":374,"threads_started":1}
I20260619 02:50:25.743872 20712 ts_manager.cc:194] Registered new tserver with Master: 9046f6d8825d48ee9f249557a32d1e38 (127.18.134.3:42225)
I20260619 02:50:25.744866 20712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.3@KRBTEST.COM'} at 127.18.134.3:34309
I20260619 02:50:25.746147 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 02:50:25 dist-test-slave-9mr0 krb5kdc[20668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
I20260619 02:50:25.773366 20920 heartbeater.cc:503] Master 127.18.134.62:35529 was elected leader, sending a full tablet report...
Jun 19 02:50:25 dist-test-slave-9mr0 krb5kdc[20668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:25.788666 20808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:25.775860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58754 (local address 127.18.134.62:35529)
0619 02:50:25.776061 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:25.776067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:25.776330 (+ 263us) server_negotiation.cc:407] Connection header received
0619 02:50:25.776476 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:25.776480 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:25.776538 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:25.776609 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:25.777825 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.778597 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.779801 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.780208 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.780305 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:25.783941 (+ 3636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:25.783948 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:25.783953 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:25.783992 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:25.786362 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:25.786994 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:25.786998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:25.787000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:25.787115 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:25.787432 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:25.787436 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:25.787439 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:25.787727 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:25.787917 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:25.788112 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 02:50:25.788320 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 02:50:25.792466 20712 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:50:25.799540 20712 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:50:25.834836 21213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:25.811579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35976 (local address 127.18.134.1:38701)
0619 02:50:25.826949 (+ 15370us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:25.826963 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:25.826995 (+ 32us) server_negotiation.cc:407] Connection header received
0619 02:50:25.827103 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:25.827111 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:25.827470 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:25.828440 (+ 970us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:25.828594 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.829599 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.832284 (+ 2685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.833738 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.833863 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:25.834063 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:25.834379 (+ 316us) server_negotiation.cc:299] Negotiation successful
0619 02:50:25.834542 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14650,"thread_start_us":14522,"threads_started":1}
I20260619 02:50:25.836901 21212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:25.811449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:38701 (local address 127.0.0.1:35976)
0619 02:50:25.818244 (+ 6795us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:25.818284 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:25.818477 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:25.827727 (+ 9250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:25.827735 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:25.827770 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:25.828202 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:25.828226 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.829721 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.829726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:25.832107 (+ 2381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:25.832121 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.833931 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.833937 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:25.834131 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:25.834187 (+ 56us) client_negotiation.cc:769] Sending connection context
0619 02:50:25.834323 (+ 136us) client_negotiation.cc:240] Negotiation successful
0619 02:50:25.834510 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6517,"thread_start_us":15050,"threads_started":1}
I20260619 02:50:25.837452 21215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:25.812796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:42985 (local address 127.0.0.1:34522)
0619 02:50:25.830525 (+ 17729us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:25.830543 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:25.830691 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:25.831307 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:25.831311 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:25.831335 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:25.831785 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:25.831797 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.833197 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.833202 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:25.834920 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:25.834935 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.836668 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.836675 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:25.836844 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:25.836877 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 02:50:25.836967 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 02:50:25.837094 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17605,"thread_start_us":17472,"threads_started":1}
I20260619 02:50:25.838114 21216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:25.813336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34522 (local address 127.18.134.2:42985)
0619 02:50:25.830115 (+ 16779us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:25.830124 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:25.830682 (+ 558us) server_negotiation.cc:407] Connection header received
0619 02:50:25.830847 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:25.830857 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:25.831115 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:25.831307 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:25.831979 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.833065 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.835095 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.836452 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.836579 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:25.836774 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:25.837662 (+ 888us) server_negotiation.cc:299] Negotiation successful
0619 02:50:25.837854 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16618,"thread_start_us":16495,"threads_started":1}
I20260619 02:50:25.839269 21211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:25.811515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:42225 (local address 127.0.0.1:53818)
0619 02:50:25.826854 (+ 15339us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:25.826875 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:25.827013 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:25.832178 (+ 5165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:25.832183 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:25.832207 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:25.832519 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:25.832529 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.834511 (+ 1982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.834516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:25.836527 (+ 2011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:25.836540 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.838567 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.838572 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:25.838765 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:25.838788 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 02:50:25.838874 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 02:50:25.838993 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15210,"thread_start_us":6315,"threads_started":1}
I20260619 02:50:25.839386 21214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:25.811966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53818 (local address 127.18.134.3:42225)
0619 02:50:25.831252 (+ 19286us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:25.831260 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:25.831290 (+ 30us) server_negotiation.cc:407] Connection header received
0619 02:50:25.831374 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:25.831720 (+ 346us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:25.831973 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:25.832702 (+ 729us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:25.832973 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.834352 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.836912 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:25.838329 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:25.838492 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:25.838734 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:25.838933 (+ 199us) server_negotiation.cc:299] Negotiation successful
0619 02:50:25.839082 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19129,"thread_start_us":18971,"threads_started":1}
I20260619 02:50:25.841250 20850 tablet_service.cc:1467] Processing CreateTablet for tablet 2874393ca94b46dd82b260d73be9c7b1 (DEFAULT_TABLE table=test-table [id=0948f7451c394399a20539cd626e4b7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:50:25.842793 20850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2874393ca94b46dd82b260d73be9c7b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:25.842325 20989 tablet_service.cc:1467] Processing CreateTablet for tablet 2874393ca94b46dd82b260d73be9c7b1 (DEFAULT_TABLE table=test-table [id=0948f7451c394399a20539cd626e4b7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:50:25.842468 21128 tablet_service.cc:1467] Processing CreateTablet for tablet 2874393ca94b46dd82b260d73be9c7b1 (DEFAULT_TABLE table=test-table [id=0948f7451c394399a20539cd626e4b7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:50:25.843678 21128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2874393ca94b46dd82b260d73be9c7b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:25.843683 20989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2874393ca94b46dd82b260d73be9c7b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:25.855113 21217 tablet_bootstrap.cc:492] T 2874393ca94b46dd82b260d73be9c7b1 P 9046f6d8825d48ee9f249557a32d1e38: Bootstrap starting.
I20260619 02:50:25.858284 21217 tablet_bootstrap.cc:654] T 2874393ca94b46dd82b260d73be9c7b1 P 9046f6d8825d48ee9f249557a32d1e38: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:25.859274 21218 tablet_bootstrap.cc:492] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692: Bootstrap starting.
I20260619 02:50:25.859499 21217 log.cc:824] T 2874393ca94b46dd82b260d73be9c7b1 P 9046f6d8825d48ee9f249557a32d1e38: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:25.860002 21219 tablet_bootstrap.cc:492] T 2874393ca94b46dd82b260d73be9c7b1 P e1412605cc804b8eb1a5a66a30889725: Bootstrap starting.
I20260619 02:50:25.862200 21217 tablet_bootstrap.cc:492] T 2874393ca94b46dd82b260d73be9c7b1 P 9046f6d8825d48ee9f249557a32d1e38: No bootstrap required, opened a new log
I20260619 02:50:25.862448 21217 ts_tablet_manager.cc:1397] T 2874393ca94b46dd82b260d73be9c7b1 P 9046f6d8825d48ee9f249557a32d1e38: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260619 02:50:25.863386 21218 tablet_bootstrap.cc:654] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:25.863879 21219 tablet_bootstrap.cc:654] T 2874393ca94b46dd82b260d73be9c7b1 P e1412605cc804b8eb1a5a66a30889725: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:25.864670 21218 log.cc:824] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:25.864990 21219 log.cc:824] T 2874393ca94b46dd82b260d73be9c7b1 P e1412605cc804b8eb1a5a66a30889725: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:25.867893 21219 tablet_bootstrap.cc:492] T 2874393ca94b46dd82b260d73be9c7b1 P e1412605cc804b8eb1a5a66a30889725: No bootstrap required, opened a new log
I20260619 02:50:25.868220 21219 ts_tablet_manager.cc:1397] T 2874393ca94b46dd82b260d73be9c7b1 P e1412605cc804b8eb1a5a66a30889725: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260619 02:50:25.869232 21218 tablet_bootstrap.cc:492] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692: No bootstrap required, opened a new log
I20260619 02:50:25.869532 21218 ts_tablet_manager.cc:1397] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260619 02:50:25.870267 21217 raft_consensus.cc:348] T 2874393ca94b46dd82b260d73be9c7b1 P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } }
I20260619 02:50:25.870601 21217 raft_consensus.cc:374] T 2874393ca94b46dd82b260d73be9c7b1 P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:25.870750 21217 raft_consensus.cc:729] T 2874393ca94b46dd82b260d73be9c7b1 P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9046f6d8825d48ee9f249557a32d1e38, State: Initialized, Role: FOLLOWER
I20260619 02:50:25.871305 21217 consensus_queue.cc:260] T 2874393ca94b46dd82b260d73be9c7b1 P 9046f6d8825d48ee9f249557a32d1e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } }
I20260619 02:50:25.872308 21198 heartbeater.cc:503] Master 127.18.134.62:35529 was elected leader, sending a full tablet report...
I20260619 02:50:25.872838 21217 ts_tablet_manager.cc:1428] T 2874393ca94b46dd82b260d73be9c7b1 P 9046f6d8825d48ee9f249557a32d1e38: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260619 02:50:25.878154 21219 raft_consensus.cc:348] T 2874393ca94b46dd82b260d73be9c7b1 P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } }
I20260619 02:50:25.878621 21219 raft_consensus.cc:374] T 2874393ca94b46dd82b260d73be9c7b1 P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:25.878755 21219 raft_consensus.cc:729] T 2874393ca94b46dd82b260d73be9c7b1 P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1412605cc804b8eb1a5a66a30889725, State: Initialized, Role: FOLLOWER
I20260619 02:50:25.878962 21218 raft_consensus.cc:348] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } }
I20260619 02:50:25.879305 21218 raft_consensus.cc:374] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:25.879390 21218 raft_consensus.cc:729] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2beb7aca7cf48289f69387c69077692, State: Initialized, Role: FOLLOWER
I20260619 02:50:25.879408 21219 consensus_queue.cc:260] T 2874393ca94b46dd82b260d73be9c7b1 P e1412605cc804b8eb1a5a66a30889725 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } }
I20260619 02:50:25.879891 21218 consensus_queue.cc:260] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } }
I20260619 02:50:25.880880 21059 heartbeater.cc:503] Master 127.18.134.62:35529 was elected leader, sending a full tablet report...
I20260619 02:50:25.881500 21218 ts_tablet_manager.cc:1428] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692: Time spent starting tablet: real 0.012s user 0.010s sys 0.000s
I20260619 02:50:25.882236 21219 ts_tablet_manager.cc:1428] T 2874393ca94b46dd82b260d73be9c7b1 P e1412605cc804b8eb1a5a66a30889725: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
W20260619 02:50:25.990603 21199 tablet.cc:2369] T 2874393ca94b46dd82b260d73be9c7b1 P 9046f6d8825d48ee9f249557a32d1e38: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:50:26.016161 20921 tablet.cc:2369] T 2874393ca94b46dd82b260d73be9c7b1 P e1412605cc804b8eb1a5a66a30889725: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:50:26.031332 21225 raft_consensus.cc:482] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:50:26.031615 21225 raft_consensus.cc:504] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } }
I20260619 02:50:26.033171 21225 leader_election.cc:288] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1412605cc804b8eb1a5a66a30889725 (127.18.134.1:38701), 9046f6d8825d48ee9f249557a32d1e38 (127.18.134.3:42225)
W20260619 02:50:26.034067 21060 tablet.cc:2369] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:50:26.039660 21213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:26.033581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:48007 (local address 127.18.134.1:38701)
0619 02:50:26.033759 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:26.033765 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:26.034729 (+ 964us) server_negotiation.cc:407] Connection header received
0619 02:50:26.034908 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:26.034911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:26.034980 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:26.035056 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:26.035661 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.036336 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.038027 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.038801 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.038946 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:26.039016 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:26.039314 (+ 298us) server_negotiation.cc:299] Negotiation successful
0619 02:50:26.039439 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 02:50:26.039671 21214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:26.033581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:57167 (local address 127.18.134.3:42225)
0619 02:50:26.033759 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:26.033765 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:26.034358 (+ 593us) server_negotiation.cc:407] Connection header received
0619 02:50:26.034700 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:26.034707 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:26.034782 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:26.034849 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:26.035542 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.036243 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.037989 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.038779 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.038926 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:26.038999 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:26.039314 (+ 315us) server_negotiation.cc:299] Negotiation successful
0619 02:50:26.039443 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 02:50:26.039806 21227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:26.033565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:38701 (local address 127.18.134.2:48007)
0619 02:50:26.034605 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:26.034622 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:26.034774 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:26.035087 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:26.035092 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:26.035115 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:26.035499 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:26.035508 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.036473 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.036478 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:26.037869 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:26.037880 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.039016 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.039022 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:26.039151 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:26.039199 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 02:50:26.039274 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 02:50:26.039407 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":913,"mutex_wait_us":14,"thread_start_us":730,"threads_started":1}
I20260619 02:50:26.039861 21226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:26.033445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:42225 (local address 127.18.134.2:57167)
0619 02:50:26.034242 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:26.034266 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:26.034490 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:26.035060 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:26.035063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:26.035083 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:26.035401 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:26.035412 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.036372 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.036376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:26.037830 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:26.037845 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.038990 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.038999 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:26.039140 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:26.039198 (+ 58us) client_negotiation.cc:769] Sending connection context
0619 02:50:26.039276 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 02:50:26.039408 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"mutex_wait_us":25,"thread_start_us":480,"threads_started":1}
I20260619 02:50:26.040695 20870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2874393ca94b46dd82b260d73be9c7b1" candidate_uuid: "e2beb7aca7cf48289f69387c69077692" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1412605cc804b8eb1a5a66a30889725" is_pre_election: true
I20260619 02:50:26.040699 21148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2874393ca94b46dd82b260d73be9c7b1" candidate_uuid: "e2beb7aca7cf48289f69387c69077692" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9046f6d8825d48ee9f249557a32d1e38" is_pre_election: true
I20260619 02:50:26.041167 20870 raft_consensus.cc:2454] T 2874393ca94b46dd82b260d73be9c7b1 P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2beb7aca7cf48289f69387c69077692 in term 0.
I20260619 02:50:26.041168 21148 raft_consensus.cc:2454] T 2874393ca94b46dd82b260d73be9c7b1 P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2beb7aca7cf48289f69387c69077692 in term 0.
I20260619 02:50:26.041723 20941 leader_election.cc:302] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e1412605cc804b8eb1a5a66a30889725, e2beb7aca7cf48289f69387c69077692; no voters:
I20260619 02:50:26.042083 21225 raft_consensus.cc:2781] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:50:26.042202 21225 raft_consensus.cc:482] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:50:26.042294 21225 raft_consensus.cc:3037] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:26.044526 21225 raft_consensus.cc:504] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } }
I20260619 02:50:26.045086 21225 leader_election.cc:288] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692 [CANDIDATE]: Term 1 election: Requested vote from peers e1412605cc804b8eb1a5a66a30889725 (127.18.134.1:38701), 9046f6d8825d48ee9f249557a32d1e38 (127.18.134.3:42225)
I20260619 02:50:26.045565 21148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2874393ca94b46dd82b260d73be9c7b1" candidate_uuid: "e2beb7aca7cf48289f69387c69077692" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9046f6d8825d48ee9f249557a32d1e38"
I20260619 02:50:26.045579 20870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2874393ca94b46dd82b260d73be9c7b1" candidate_uuid: "e2beb7aca7cf48289f69387c69077692" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1412605cc804b8eb1a5a66a30889725"
I20260619 02:50:26.045735 20870 raft_consensus.cc:3037] T 2874393ca94b46dd82b260d73be9c7b1 P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:26.045744 21148 raft_consensus.cc:3037] T 2874393ca94b46dd82b260d73be9c7b1 P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:26.047809 20870 raft_consensus.cc:2454] T 2874393ca94b46dd82b260d73be9c7b1 P e1412605cc804b8eb1a5a66a30889725 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2beb7aca7cf48289f69387c69077692 in term 1.
I20260619 02:50:26.047932 21148 raft_consensus.cc:2454] T 2874393ca94b46dd82b260d73be9c7b1 P 9046f6d8825d48ee9f249557a32d1e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2beb7aca7cf48289f69387c69077692 in term 1.
I20260619 02:50:26.048192 20941 leader_election.cc:302] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e1412605cc804b8eb1a5a66a30889725, e2beb7aca7cf48289f69387c69077692; no voters:
I20260619 02:50:26.048466 21225 raft_consensus.cc:2781] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:50:26.048871 21225 raft_consensus.cc:686] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692 [term 1 LEADER]: Becoming Leader. State: Replica: e2beb7aca7cf48289f69387c69077692, State: Running, Role: LEADER
I20260619 02:50:26.049427 21225 consensus_queue.cc:237] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } }
I20260619 02:50:26.053702 20713 catalog_manager.cc:5495] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2beb7aca7cf48289f69387c69077692 (127.18.134.2). New cstate: current_term: 1 leader_uuid: "e2beb7aca7cf48289f69387c69077692" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } health_report { overall_health: HEALTHY } } }
Jun 19 02:50:26 dist-test-slave-9mr0 krb5kdc[20668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837424, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:26.102453 20808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:26.085188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58762 (local address 127.18.134.62:35529)
0619 02:50:26.085396 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:26.085401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:26.085991 (+ 590us) server_negotiation.cc:407] Connection header received
0619 02:50:26.086123 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:26.086127 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:26.086247 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:26.086340 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:26.087544 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.088332 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.089665 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.090077 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.090173 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:26.097011 (+ 6838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:26.097016 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:26.097019 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:26.097052 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:26.099645 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:26.100523 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:26.100527 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:26.100530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:26.100647 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:26.100994 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:26.100998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:26.101000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:26.101273 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:26.101399 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:26.101912 (+ 513us) server_negotiation.cc:299] Negotiation successful
0619 02:50:26.102079 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 02:50:26.102458 21235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:26.085343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35529 (local address 127.0.0.1:58762)
0619 02:50:26.085883 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:26.085903 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:26.086027 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:26.086399 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:26.086402 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:26.087108 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:26.087379 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:26.087389 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.088480 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.088485 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:26.089530 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:26.089543 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.090232 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.090237 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:26.090403 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:26.094613 (+ 4210us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:26.094682 (+ 69us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:26.096812 (+ 2130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:26.099839 (+ 3027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:26.099855 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:26.099862 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:26.100400 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:26.100743 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:26.100747 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:26.100749 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:26.100899 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:26.101425 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:26.101432 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:26.101601 (+ 169us) client_negotiation.cc:769] Sending connection context
0619 02:50:26.101752 (+ 151us) client_negotiation.cc:240] Negotiation successful
0619 02:50:26.101976 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":282,"threads_started":1}
I20260619 02:50:26.107327 20713 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:58762:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 02:50:26.108366 20713 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:50:26.114671 20850 tablet_service.cc:1467] Processing CreateTablet for tablet df44e1afa23e4209818f921aa5d1a6dc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=038ca6341199444699b4a0b1a3763c1c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:50:26.114705 20989 tablet_service.cc:1467] Processing CreateTablet for tablet df44e1afa23e4209818f921aa5d1a6dc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=038ca6341199444699b4a0b1a3763c1c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:50:26.114923 21128 tablet_service.cc:1467] Processing CreateTablet for tablet df44e1afa23e4209818f921aa5d1a6dc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=038ca6341199444699b4a0b1a3763c1c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:50:26.115675 20989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df44e1afa23e4209818f921aa5d1a6dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:26.115676 20850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df44e1afa23e4209818f921aa5d1a6dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:26.115760 21128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df44e1afa23e4209818f921aa5d1a6dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:26.121474 21217 tablet_bootstrap.cc:492] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38: Bootstrap starting.
I20260619 02:50:26.122505 21219 tablet_bootstrap.cc:492] T df44e1afa23e4209818f921aa5d1a6dc P e1412605cc804b8eb1a5a66a30889725: Bootstrap starting.
I20260619 02:50:26.122756 21218 tablet_bootstrap.cc:492] T df44e1afa23e4209818f921aa5d1a6dc P e2beb7aca7cf48289f69387c69077692: Bootstrap starting.
I20260619 02:50:26.122960 21217 tablet_bootstrap.cc:654] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:26.124038 21219 tablet_bootstrap.cc:654] T df44e1afa23e4209818f921aa5d1a6dc P e1412605cc804b8eb1a5a66a30889725: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:26.124169 21218 tablet_bootstrap.cc:654] T df44e1afa23e4209818f921aa5d1a6dc P e2beb7aca7cf48289f69387c69077692: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:26.126238 21218 tablet_bootstrap.cc:492] T df44e1afa23e4209818f921aa5d1a6dc P e2beb7aca7cf48289f69387c69077692: No bootstrap required, opened a new log
I20260619 02:50:26.126376 21218 ts_tablet_manager.cc:1397] T df44e1afa23e4209818f921aa5d1a6dc P e2beb7aca7cf48289f69387c69077692: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 02:50:26.126454 21219 tablet_bootstrap.cc:492] T df44e1afa23e4209818f921aa5d1a6dc P e1412605cc804b8eb1a5a66a30889725: No bootstrap required, opened a new log
I20260619 02:50:26.126590 21219 ts_tablet_manager.cc:1397] T df44e1afa23e4209818f921aa5d1a6dc P e1412605cc804b8eb1a5a66a30889725: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260619 02:50:26.126655 21217 tablet_bootstrap.cc:492] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38: No bootstrap required, opened a new log
I20260619 02:50:26.126780 21217 ts_tablet_manager.cc:1397] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.001s
I20260619 02:50:26.127079 21218 raft_consensus.cc:348] T df44e1afa23e4209818f921aa5d1a6dc P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:26.127249 21218 raft_consensus.cc:374] T df44e1afa23e4209818f921aa5d1a6dc P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:26.127326 21218 raft_consensus.cc:729] T df44e1afa23e4209818f921aa5d1a6dc P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2beb7aca7cf48289f69387c69077692, State: Initialized, Role: FOLLOWER
I20260619 02:50:26.127336 21219 raft_consensus.cc:348] T df44e1afa23e4209818f921aa5d1a6dc P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:26.127553 21219 raft_consensus.cc:374] T df44e1afa23e4209818f921aa5d1a6dc P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:26.127629 21219 raft_consensus.cc:729] T df44e1afa23e4209818f921aa5d1a6dc P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1412605cc804b8eb1a5a66a30889725, State: Initialized, Role: FOLLOWER
I20260619 02:50:26.127496 21217 raft_consensus.cc:348] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:26.127526 21218 consensus_queue.cc:260] T df44e1afa23e4209818f921aa5d1a6dc P e2beb7aca7cf48289f69387c69077692 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:26.127701 21217 raft_consensus.cc:374] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:26.127835 21217 raft_consensus.cc:729] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9046f6d8825d48ee9f249557a32d1e38, State: Initialized, Role: FOLLOWER
I20260619 02:50:26.127848 21219 consensus_queue.cc:260] T df44e1afa23e4209818f921aa5d1a6dc P e1412605cc804b8eb1a5a66a30889725 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:26.128080 21218 ts_tablet_manager.cc:1428] T df44e1afa23e4209818f921aa5d1a6dc P e2beb7aca7cf48289f69387c69077692: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:50:26.128032 21217 consensus_queue.cc:260] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:26.128376 21219 ts_tablet_manager.cc:1428] T df44e1afa23e4209818f921aa5d1a6dc P e1412605cc804b8eb1a5a66a30889725: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:50:26.128400 21225 tablet_replica.cc:439] T df44e1afa23e4209818f921aa5d1a6dc P e2beb7aca7cf48289f69387c69077692: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } } }
I20260619 02:50:26.128661 21225 tablet_replica.cc:442] T df44e1afa23e4209818f921aa5d1a6dc P e2beb7aca7cf48289f69387c69077692: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:50:26.128535 21223 tablet_replica.cc:439] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } } }
I20260619 02:50:26.128858 21223 tablet_replica.cc:442] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:50:26.129163 21217 ts_tablet_manager.cc:1428] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:50:26.128993 21224 tablet_replica.cc:439] T df44e1afa23e4209818f921aa5d1a6dc P e1412605cc804b8eb1a5a66a30889725: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } } }
I20260619 02:50:26.129319 21224 tablet_replica.cc:442] T df44e1afa23e4209818f921aa5d1a6dc P e1412605cc804b8eb1a5a66a30889725: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:50:26.289825 21223 raft_consensus.cc:482] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:50:26.290030 21223 raft_consensus.cc:504] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:26.291524 21223 leader_election.cc:288] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1412605cc804b8eb1a5a66a30889725 (127.18.134.1:38701), e2beb7aca7cf48289f69387c69077692 (127.18.134.2:42985)
I20260619 02:50:26.297833 21216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:26.291926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:55745 (local address 127.18.134.2:42985)
0619 02:50:26.292111 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:26.292117 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:26.292529 (+ 412us) server_negotiation.cc:407] Connection header received
0619 02:50:26.292707 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:26.292711 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:26.292771 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:26.292869 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:26.293350 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.294252 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.296025 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.296937 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.297059 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:26.297111 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:26.297457 (+ 346us) server_negotiation.cc:299] Negotiation successful
0619 02:50:26.297579 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 02:50:26.297845 21239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:26.291848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:42985 (local address 127.18.134.3:55745)
0619 02:50:26.292369 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:26.292430 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:26.292591 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:26.292932 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:26.292937 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:26.292973 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:26.293193 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:26.293204 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.294377 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.294383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:26.295853 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:26.295865 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.297091 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.297098 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:26.297295 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:26.297329 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 02:50:26.297429 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 02:50:26.297569 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":262,"threads_started":1}
I20260619 02:50:26.298290 21213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:26.292061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:38509 (local address 127.18.134.1:38701)
0619 02:50:26.292219 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:26.292223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:26.293097 (+ 874us) server_negotiation.cc:407] Connection header received
0619 02:50:26.293227 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:26.293232 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:26.293305 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:26.293376 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:26.293873 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.294746 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.296472 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.297359 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.297514 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:26.297579 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:26.297921 (+ 342us) server_negotiation.cc:299] Negotiation successful
0619 02:50:26.298048 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 02:50:26.298347 21240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:26.291857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:38701 (local address 127.18.134.3:38509)
0619 02:50:26.292998 (+ 1141us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:26.293014 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:26.293106 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:26.293420 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:26.293424 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:26.293449 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:26.293733 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:26.293744 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.294869 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.294876 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:26.296322 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:26.296336 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.297572 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.297578 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:26.297740 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:26.297769 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 02:50:26.297866 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 02:50:26.298003 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1066,"mutex_wait_us":61,"thread_start_us":846,"threads_started":1}
I20260619 02:50:26.298985 20870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df44e1afa23e4209818f921aa5d1a6dc" candidate_uuid: "9046f6d8825d48ee9f249557a32d1e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1412605cc804b8eb1a5a66a30889725" is_pre_election: true
I20260619 02:50:26.298952 21009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df44e1afa23e4209818f921aa5d1a6dc" candidate_uuid: "9046f6d8825d48ee9f249557a32d1e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2beb7aca7cf48289f69387c69077692" is_pre_election: true
I20260619 02:50:26.299230 20870 raft_consensus.cc:2454] T df44e1afa23e4209818f921aa5d1a6dc P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9046f6d8825d48ee9f249557a32d1e38 in term 0.
I20260619 02:50:26.299351 21009 raft_consensus.cc:2454] T df44e1afa23e4209818f921aa5d1a6dc P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9046f6d8825d48ee9f249557a32d1e38 in term 0.
I20260619 02:50:26.299746 21080 leader_election.cc:302] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9046f6d8825d48ee9f249557a32d1e38, e1412605cc804b8eb1a5a66a30889725; no voters:
I20260619 02:50:26.300173 21223 raft_consensus.cc:2781] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:50:26.300288 21223 raft_consensus.cc:482] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:50:26.300371 21223 raft_consensus.cc:3037] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:26.302388 21223 raft_consensus.cc:504] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:26.303030 21223 leader_election.cc:288] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38 [CANDIDATE]: Term 1 election: Requested vote from peers e1412605cc804b8eb1a5a66a30889725 (127.18.134.1:38701), e2beb7aca7cf48289f69387c69077692 (127.18.134.2:42985)
I20260619 02:50:26.303424 20870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df44e1afa23e4209818f921aa5d1a6dc" candidate_uuid: "9046f6d8825d48ee9f249557a32d1e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1412605cc804b8eb1a5a66a30889725"
I20260619 02:50:26.303462 21009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df44e1afa23e4209818f921aa5d1a6dc" candidate_uuid: "9046f6d8825d48ee9f249557a32d1e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2beb7aca7cf48289f69387c69077692"
I20260619 02:50:26.303614 20870 raft_consensus.cc:3037] T df44e1afa23e4209818f921aa5d1a6dc P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:26.303617 21009 raft_consensus.cc:3037] T df44e1afa23e4209818f921aa5d1a6dc P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:26.305796 20870 raft_consensus.cc:2454] T df44e1afa23e4209818f921aa5d1a6dc P e1412605cc804b8eb1a5a66a30889725 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9046f6d8825d48ee9f249557a32d1e38 in term 1.
I20260619 02:50:26.305931 21009 raft_consensus.cc:2454] T df44e1afa23e4209818f921aa5d1a6dc P e2beb7aca7cf48289f69387c69077692 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9046f6d8825d48ee9f249557a32d1e38 in term 1.
I20260619 02:50:26.306147 21080 leader_election.cc:302] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9046f6d8825d48ee9f249557a32d1e38, e1412605cc804b8eb1a5a66a30889725; no voters:
I20260619 02:50:26.306420 21223 raft_consensus.cc:2781] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:50:26.306794 21223 raft_consensus.cc:686] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38 [term 1 LEADER]: Becoming Leader. State: Replica: 9046f6d8825d48ee9f249557a32d1e38, State: Running, Role: LEADER
I20260619 02:50:26.307264 21223 consensus_queue.cc:237] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:26.310528 21241 tablet_replica.cc:439] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38: TxnStatusTablet state changed. Reason: New leader 9046f6d8825d48ee9f249557a32d1e38. Latest consensus state: current_term: 1 leader_uuid: "9046f6d8825d48ee9f249557a32d1e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } } }
I20260619 02:50:26.310949 21241 tablet_replica.cc:442] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:50:26.311560 21243 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 02:50:26.311784 21243 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 02:50:26.311739 20713 catalog_manager.cc:5495] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9046f6d8825d48ee9f249557a32d1e38 (127.18.134.3). New cstate: current_term: 1 leader_uuid: "9046f6d8825d48ee9f249557a32d1e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } health_report { overall_health: HEALTHY } } }
I20260619 02:50:26.364081 21214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:26.358851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53832 (local address 127.18.134.3:42225)
0619 02:50:26.359022 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:26.359029 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:26.359073 (+ 44us) server_negotiation.cc:407] Connection header received
0619 02:50:26.359354 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:26.359358 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:26.359467 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:26.359591 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:50:26.359965 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.360767 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.361975 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.362368 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.362475 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:26.362925 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:50:26.363443 (+ 518us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:50:26.363566 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:26.363667 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 02:50:26.363750 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 02:50:26.364078 21235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:26.358759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:42225 (local address 127.0.0.1:53832)
0619 02:50:26.359002 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:26.359021 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:26.359154 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:26.359623 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:26.359628 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:26.359638 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:50:26.359817 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:26.359825 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.360886 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.360892 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:26.361843 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:26.361854 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.362511 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.362517 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:26.362696 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:26.362757 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:50:26.363545 (+ 788us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:50:26.363573 (+ 28us) client_negotiation.cc:769] Sending connection context
0619 02:50:26.363646 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 02:50:26.363805 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 02:50:26.371354 20870 raft_consensus.cc:1261] T df44e1afa23e4209818f921aa5d1a6dc P e1412605cc804b8eb1a5a66a30889725 [term 1 FOLLOWER]: Refusing update from remote peer 9046f6d8825d48ee9f249557a32d1e38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:50:26.371337 21009 raft_consensus.cc:1261] T df44e1afa23e4209818f921aa5d1a6dc P e2beb7aca7cf48289f69387c69077692 [term 1 FOLLOWER]: Refusing update from remote peer 9046f6d8825d48ee9f249557a32d1e38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:50:26.372320 21241 consensus_queue.cc:1035] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:50:26.373025 21223 consensus_queue.cc:1035] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:50:26.383496 21225 tablet_replica.cc:439] T df44e1afa23e4209818f921aa5d1a6dc P e2beb7aca7cf48289f69387c69077692: TxnStatusTablet state changed. Reason: New leader 9046f6d8825d48ee9f249557a32d1e38. Latest consensus state: current_term: 1 leader_uuid: "9046f6d8825d48ee9f249557a32d1e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } } }
I20260619 02:50:26.383867 21225 tablet_replica.cc:442] T df44e1afa23e4209818f921aa5d1a6dc P e2beb7aca7cf48289f69387c69077692: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:50:26.388777 21241 tablet_replica.cc:439] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9046f6d8825d48ee9f249557a32d1e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } } }
I20260619 02:50:26.389084 21241 tablet_replica.cc:442] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:50:26.396679 21224 tablet_replica.cc:439] T df44e1afa23e4209818f921aa5d1a6dc P e1412605cc804b8eb1a5a66a30889725: TxnStatusTablet state changed. Reason: New leader 9046f6d8825d48ee9f249557a32d1e38. Latest consensus state: current_term: 1 leader_uuid: "9046f6d8825d48ee9f249557a32d1e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } } }
I20260619 02:50:26.396761 21223 tablet_replica.cc:439] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9046f6d8825d48ee9f249557a32d1e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } } }
I20260619 02:50:26.396966 21224 tablet_replica.cc:442] T df44e1afa23e4209818f921aa5d1a6dc P e1412605cc804b8eb1a5a66a30889725: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:50:26.397029 21223 tablet_replica.cc:442] T df44e1afa23e4209818f921aa5d1a6dc P 9046f6d8825d48ee9f249557a32d1e38: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:50:26.398962 21248 mvcc.cc:205] Tried to move back new op lower bound from 7298406098407514112 to 7298406098158288896. Current Snapshot: MvccSnapshot[applied={T|T < 7298406098407514112}]
I20260619 02:50:26.399920 21224 tablet_replica.cc:439] T df44e1afa23e4209818f921aa5d1a6dc P e1412605cc804b8eb1a5a66a30889725: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9046f6d8825d48ee9f249557a32d1e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } } }
I20260619 02:50:26.400146 21224 tablet_replica.cc:442] T df44e1afa23e4209818f921aa5d1a6dc P e1412605cc804b8eb1a5a66a30889725: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:50:26.400511 21225 tablet_replica.cc:439] T df44e1afa23e4209818f921aa5d1a6dc P e2beb7aca7cf48289f69387c69077692: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9046f6d8825d48ee9f249557a32d1e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } } }
I20260619 02:50:26.400758 21225 tablet_replica.cc:442] T df44e1afa23e4209818f921aa5d1a6dc P e2beb7aca7cf48289f69387c69077692: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:50:26.414094 21216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:26.407627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34524 (local address 127.18.134.2:42985)
0619 02:50:26.407804 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:26.407810 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:26.408269 (+ 459us) server_negotiation.cc:407] Connection header received
0619 02:50:26.408461 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:26.408467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:26.408574 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:26.408762 (+ 188us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:50:26.409123 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.410225 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.411631 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.412256 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.412382 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:26.412776 (+ 394us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:50:26.413479 (+ 703us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:50:26.413642 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:26.413696 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 02:50:26.413778 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 02:50:26.431790 21258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:26.425507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:42225 (local address 127.18.134.2:43195)
0619 02:50:26.426042 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:26.426057 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:26.426215 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:26.426709 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:26.426714 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:26.426726 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:50:26.426995 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:26.427005 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.428442 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.428446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:26.429646 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:26.429658 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.430594 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.430599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:26.430758 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:26.430809 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:50:26.431315 (+ 506us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:50:26.431335 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 02:50:26.431412 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 02:50:26.431513 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":293,"threads_started":1}
I20260619 02:50:26.431816 21214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:26.425580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:43195 (local address 127.18.134.3:42225)
0619 02:50:26.425742 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:26.425752 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:26.426157 (+ 405us) server_negotiation.cc:407] Connection header received
0619 02:50:26.426345 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:26.426351 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:26.426470 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:26.426559 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:50:26.427168 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.428318 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.429772 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.430446 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.430569 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:26.431004 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:50:26.431146 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:50:26.431238 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:26.431455 (+ 217us) server_negotiation.cc:299] Negotiation successful
0619 02:50:26.431531 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 02:50:26.441121 20870 raft_consensus.cc:1261] T 2874393ca94b46dd82b260d73be9c7b1 P e1412605cc804b8eb1a5a66a30889725 [term 1 FOLLOWER]: Refusing update from remote peer e2beb7aca7cf48289f69387c69077692: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:50:26.441394 21148 raft_consensus.cc:1261] T 2874393ca94b46dd82b260d73be9c7b1 P 9046f6d8825d48ee9f249557a32d1e38 [term 1 FOLLOWER]: Refusing update from remote peer e2beb7aca7cf48289f69387c69077692: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:50:26.441748 21228 consensus_queue.cc:1035] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:50:26.442112 21225 consensus_queue.cc:1035] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:50:26.447930 21247 mvcc.cc:205] Tried to move back new op lower bound from 7298406098696941568 to 7298406097101885440. Current Snapshot: MvccSnapshot[applied={T|T < 7298406098696941568}]
I20260619 02:50:26.458297 21246 mvcc.cc:205] Tried to move back new op lower bound from 7298406098696941568 to 7298406097101885440. Current Snapshot: MvccSnapshot[applied={T|T < 7298406098696941568}]
I20260619 02:50:26.483997 21216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:26.477028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:41129 (local address 127.18.134.2:42985)
0619 02:50:26.477219 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:26.477226 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:26.477808 (+ 582us) server_negotiation.cc:407] Connection header received
0619 02:50:26.477996 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:26.478001 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:26.478084 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:26.478171 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:50:26.478796 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.479605 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.482103 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.482732 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.482847 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:26.483372 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:50:26.483510 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:50:26.483575 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:26.483713 (+ 138us) server_negotiation.cc:299] Negotiation successful
0619 02:50:26.483790 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 02:50:26.484264 21264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:26.477025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:42985 (local address 127.18.134.3:41129)
0619 02:50:26.477714 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:26.477730 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:26.477864 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:26.478194 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:26.478200 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:26.478213 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:50:26.478625 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:26.478662 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.479733 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.479738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:26.481970 (+ 2232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:26.481985 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.482932 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.482939 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:26.483112 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:26.483186 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:50:26.483602 (+ 416us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:50:26.483614 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 02:50:26.483689 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 02:50:26.483855 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":340,"threads_started":1}
I20260619 02:50:26.521067 20713 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:50:26.521355 20713 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:50:26.525388 20713 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6a59ebf29ad4eeab57577c61122f952: Sending DeleteTablet for 3 replicas of tablet 2874393ca94b46dd82b260d73be9c7b1
I20260619 02:50:26.526834 20989 tablet_service.cc:1514] Processing DeleteTablet for tablet 2874393ca94b46dd82b260d73be9c7b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:50:26 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:34522
I20260619 02:50:26.526858 20850 tablet_service.cc:1514] Processing DeleteTablet for tablet 2874393ca94b46dd82b260d73be9c7b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:50:26 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:35976
I20260619 02:50:26.528086 21128 tablet_service.cc:1514] Processing DeleteTablet for tablet 2874393ca94b46dd82b260d73be9c7b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:50:26 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:53818
I20260619 02:50:26.532846 20713 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:50:26.533586 20713 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:50:26.539172 21269 tablet_replica.cc:330] T 2874393ca94b46dd82b260d73be9c7b1 P 9046f6d8825d48ee9f249557a32d1e38: stopping tablet replica
I20260619 02:50:26.539634 21269 raft_consensus.cc:2229] T 2874393ca94b46dd82b260d73be9c7b1 P 9046f6d8825d48ee9f249557a32d1e38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:50:26.540021 21269 raft_consensus.cc:2258] T 2874393ca94b46dd82b260d73be9c7b1 P 9046f6d8825d48ee9f249557a32d1e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:50:26.542924 21269 ts_tablet_manager.cc:1905] T 2874393ca94b46dd82b260d73be9c7b1 P 9046f6d8825d48ee9f249557a32d1e38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:50:26.546147 21268 tablet_replica.cc:330] T 2874393ca94b46dd82b260d73be9c7b1 P e1412605cc804b8eb1a5a66a30889725: stopping tablet replica
I20260619 02:50:26.546301 20849 tablet_service.cc:1467] Processing CreateTablet for tablet 8a5ec0be444d4d8fa5cfef871812cc4a (DEFAULT_TABLE table=test-table [id=02227f7c12234fd08c5d5a87a1879da6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:50:26.546387 20988 tablet_service.cc:1467] Processing CreateTablet for tablet 8a5ec0be444d4d8fa5cfef871812cc4a (DEFAULT_TABLE table=test-table [id=02227f7c12234fd08c5d5a87a1879da6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:50:26.546604 21268 raft_consensus.cc:2229] T 2874393ca94b46dd82b260d73be9c7b1 P e1412605cc804b8eb1a5a66a30889725 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:50:26.546883 20988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a5ec0be444d4d8fa5cfef871812cc4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:26.546926 20849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a5ec0be444d4d8fa5cfef871812cc4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:26.547008 21268 raft_consensus.cc:2258] T 2874393ca94b46dd82b260d73be9c7b1 P e1412605cc804b8eb1a5a66a30889725 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:50:26.546850 21128 tablet_service.cc:1467] Processing CreateTablet for tablet 8a5ec0be444d4d8fa5cfef871812cc4a (DEFAULT_TABLE table=test-table [id=02227f7c12234fd08c5d5a87a1879da6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:50:26.547384 21128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a5ec0be444d4d8fa5cfef871812cc4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:26.547711 21267 tablet_replica.cc:330] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692: stopping tablet replica
I20260619 02:50:26.548112 21267 raft_consensus.cc:2229] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:50:26.548705 21267 raft_consensus.cc:2258] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:50:26.550659 21268 ts_tablet_manager.cc:1905] T 2874393ca94b46dd82b260d73be9c7b1 P e1412605cc804b8eb1a5a66a30889725: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:50:26.553026 21218 tablet_bootstrap.cc:492] T 8a5ec0be444d4d8fa5cfef871812cc4a P e2beb7aca7cf48289f69387c69077692: Bootstrap starting.
I20260619 02:50:26.554541 21218 tablet_bootstrap.cc:654] T 8a5ec0be444d4d8fa5cfef871812cc4a P e2beb7aca7cf48289f69387c69077692: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:26.556475 21219 tablet_bootstrap.cc:492] T 8a5ec0be444d4d8fa5cfef871812cc4a P e1412605cc804b8eb1a5a66a30889725: Bootstrap starting.
I20260619 02:50:26.557124 21217 tablet_bootstrap.cc:492] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38: Bootstrap starting.
I20260619 02:50:26.558174 21219 tablet_bootstrap.cc:654] T 8a5ec0be444d4d8fa5cfef871812cc4a P e1412605cc804b8eb1a5a66a30889725: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:26.558357 21217 tablet_bootstrap.cc:654] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:26.560595 21267 ts_tablet_manager.cc:1905] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:50:26.568161 21217 tablet_bootstrap.cc:492] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38: No bootstrap required, opened a new log
I20260619 02:50:26.568341 21217 ts_tablet_manager.cc:1397] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.010s
I20260619 02:50:26.569136 21217 raft_consensus.cc:348] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:26.569360 21217 raft_consensus.cc:374] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:26.569437 21217 raft_consensus.cc:729] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9046f6d8825d48ee9f249557a32d1e38, State: Initialized, Role: FOLLOWER
I20260619 02:50:26.569684 21217 consensus_queue.cc:260] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:26.571735 21269 ts_tablet_manager.cc:1918] T 2874393ca94b46dd82b260d73be9c7b1 P 9046f6d8825d48ee9f249557a32d1e38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:50:26.571918 21269 log.cc:1196] T 2874393ca94b46dd82b260d73be9c7b1 P 9046f6d8825d48ee9f249557a32d1e38: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-2/wal/wals/2874393ca94b46dd82b260d73be9c7b1
I20260619 02:50:26.572427 21269 ts_tablet_manager.cc:1939] T 2874393ca94b46dd82b260d73be9c7b1 P 9046f6d8825d48ee9f249557a32d1e38: Deleting consensus metadata
I20260619 02:50:26.572520 21217 ts_tablet_manager.cc:1428] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260619 02:50:26.574177 21219 tablet_bootstrap.cc:492] T 8a5ec0be444d4d8fa5cfef871812cc4a P e1412605cc804b8eb1a5a66a30889725: No bootstrap required, opened a new log
I20260619 02:50:26.574359 21219 ts_tablet_manager.cc:1397] T 8a5ec0be444d4d8fa5cfef871812cc4a P e1412605cc804b8eb1a5a66a30889725: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.000s
I20260619 02:50:26.574375 21268 ts_tablet_manager.cc:1918] T 2874393ca94b46dd82b260d73be9c7b1 P e1412605cc804b8eb1a5a66a30889725: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:50:26.575333 21268 log.cc:1196] T 2874393ca94b46dd82b260d73be9c7b1 P e1412605cc804b8eb1a5a66a30889725: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-0/wal/wals/2874393ca94b46dd82b260d73be9c7b1
I20260619 02:50:26.575371 21219 raft_consensus.cc:348] T 8a5ec0be444d4d8fa5cfef871812cc4a P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:26.575603 21219 raft_consensus.cc:374] T 8a5ec0be444d4d8fa5cfef871812cc4a P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:26.575680 21219 raft_consensus.cc:729] T 8a5ec0be444d4d8fa5cfef871812cc4a P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1412605cc804b8eb1a5a66a30889725, State: Initialized, Role: FOLLOWER
I20260619 02:50:26.576304 21218 tablet_bootstrap.cc:492] T 8a5ec0be444d4d8fa5cfef871812cc4a P e2beb7aca7cf48289f69387c69077692: No bootstrap required, opened a new log
I20260619 02:50:26.576540 21218 ts_tablet_manager.cc:1397] T 8a5ec0be444d4d8fa5cfef871812cc4a P e2beb7aca7cf48289f69387c69077692: Time spent bootstrapping tablet: real 0.024s user 0.007s sys 0.004s
I20260619 02:50:26.577334 21218 raft_consensus.cc:348] T 8a5ec0be444d4d8fa5cfef871812cc4a P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:26.577579 21218 raft_consensus.cc:374] T 8a5ec0be444d4d8fa5cfef871812cc4a P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:26.577662 21218 raft_consensus.cc:729] T 8a5ec0be444d4d8fa5cfef871812cc4a P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2beb7aca7cf48289f69387c69077692, State: Initialized, Role: FOLLOWER
I20260619 02:50:26.577734 20699 catalog_manager.cc:4841] TS 9046f6d8825d48ee9f249557a32d1e38 (127.18.134.3:42225): tablet 2874393ca94b46dd82b260d73be9c7b1 (table test-table [id=0948f7451c394399a20539cd626e4b7d]) successfully deleted
I20260619 02:50:26.577926 21218 consensus_queue.cc:260] T 8a5ec0be444d4d8fa5cfef871812cc4a P e2beb7aca7cf48289f69387c69077692 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:26.578936 21267 ts_tablet_manager.cc:1918] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:50:26.579101 21267 log.cc:1196] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-1/wal/wals/2874393ca94b46dd82b260d73be9c7b1
I20260619 02:50:26.579576 21267 ts_tablet_manager.cc:1939] T 2874393ca94b46dd82b260d73be9c7b1 P e2beb7aca7cf48289f69387c69077692: Deleting consensus metadata
I20260619 02:50:26.580845 21218 ts_tablet_manager.cc:1428] T 8a5ec0be444d4d8fa5cfef871812cc4a P e2beb7aca7cf48289f69387c69077692: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260619 02:50:26.581244 20697 catalog_manager.cc:4841] TS e2beb7aca7cf48289f69387c69077692 (127.18.134.2:42985): tablet 2874393ca94b46dd82b260d73be9c7b1 (table test-table [id=0948f7451c394399a20539cd626e4b7d]) successfully deleted
I20260619 02:50:26.581653 21219 consensus_queue.cc:260] T 8a5ec0be444d4d8fa5cfef871812cc4a P e1412605cc804b8eb1a5a66a30889725 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:26.581982 21268 ts_tablet_manager.cc:1939] T 2874393ca94b46dd82b260d73be9c7b1 P e1412605cc804b8eb1a5a66a30889725: Deleting consensus metadata
I20260619 02:50:26.582305 21219 ts_tablet_manager.cc:1428] T 8a5ec0be444d4d8fa5cfef871812cc4a P e1412605cc804b8eb1a5a66a30889725: Time spent starting tablet: real 0.008s user 0.000s sys 0.002s
I20260619 02:50:26.584030 20698 catalog_manager.cc:4841] TS e1412605cc804b8eb1a5a66a30889725 (127.18.134.1:38701): tablet 2874393ca94b46dd82b260d73be9c7b1 (table test-table [id=0948f7451c394399a20539cd626e4b7d]) successfully deleted
I20260619 02:50:26.817685 21241 raft_consensus.cc:482] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:50:26.817883 21241 raft_consensus.cc:504] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:26.818490 21241 leader_election.cc:288] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1412605cc804b8eb1a5a66a30889725 (127.18.134.1:38701), e2beb7aca7cf48289f69387c69077692 (127.18.134.2:42985)
I20260619 02:50:26.819119 20870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a5ec0be444d4d8fa5cfef871812cc4a" candidate_uuid: "9046f6d8825d48ee9f249557a32d1e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1412605cc804b8eb1a5a66a30889725" is_pre_election: true
I20260619 02:50:26.819121 21009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a5ec0be444d4d8fa5cfef871812cc4a" candidate_uuid: "9046f6d8825d48ee9f249557a32d1e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2beb7aca7cf48289f69387c69077692" is_pre_election: true
I20260619 02:50:26.819365 20870 raft_consensus.cc:2454] T 8a5ec0be444d4d8fa5cfef871812cc4a P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9046f6d8825d48ee9f249557a32d1e38 in term 0.
I20260619 02:50:26.819372 21009 raft_consensus.cc:2454] T 8a5ec0be444d4d8fa5cfef871812cc4a P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9046f6d8825d48ee9f249557a32d1e38 in term 0.
I20260619 02:50:26.819813 21079 leader_election.cc:302] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9046f6d8825d48ee9f249557a32d1e38, e2beb7aca7cf48289f69387c69077692; no voters:
I20260619 02:50:26.820111 21241 raft_consensus.cc:2781] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:50:26.820199 21241 raft_consensus.cc:482] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:50:26.820300 21241 raft_consensus.cc:3037] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:26.822291 21241 raft_consensus.cc:504] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:26.822911 21241 leader_election.cc:288] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38 [CANDIDATE]: Term 1 election: Requested vote from peers e1412605cc804b8eb1a5a66a30889725 (127.18.134.1:38701), e2beb7aca7cf48289f69387c69077692 (127.18.134.2:42985)
I20260619 02:50:26.823405 20870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a5ec0be444d4d8fa5cfef871812cc4a" candidate_uuid: "9046f6d8825d48ee9f249557a32d1e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1412605cc804b8eb1a5a66a30889725"
I20260619 02:50:26.823438 21009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a5ec0be444d4d8fa5cfef871812cc4a" candidate_uuid: "9046f6d8825d48ee9f249557a32d1e38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2beb7aca7cf48289f69387c69077692"
I20260619 02:50:26.823581 20870 raft_consensus.cc:3037] T 8a5ec0be444d4d8fa5cfef871812cc4a P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:26.823683 21009 raft_consensus.cc:3037] T 8a5ec0be444d4d8fa5cfef871812cc4a P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:26.825527 20870 raft_consensus.cc:2454] T 8a5ec0be444d4d8fa5cfef871812cc4a P e1412605cc804b8eb1a5a66a30889725 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9046f6d8825d48ee9f249557a32d1e38 in term 1.
I20260619 02:50:26.825903 21080 leader_election.cc:302] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9046f6d8825d48ee9f249557a32d1e38, e1412605cc804b8eb1a5a66a30889725; no voters:
I20260619 02:50:26.826136 21241 raft_consensus.cc:2781] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:50:26.826380 21241 raft_consensus.cc:686] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38 [term 1 LEADER]: Becoming Leader. State: Replica: 9046f6d8825d48ee9f249557a32d1e38, State: Running, Role: LEADER
I20260619 02:50:26.826681 21241 consensus_queue.cc:237] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:26.827035 21009 raft_consensus.cc:2454] T 8a5ec0be444d4d8fa5cfef871812cc4a P e2beb7aca7cf48289f69387c69077692 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9046f6d8825d48ee9f249557a32d1e38 in term 1.
I20260619 02:50:26.829656 20713 catalog_manager.cc:5495] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9046f6d8825d48ee9f249557a32d1e38 (127.18.134.3). New cstate: current_term: 1 leader_uuid: "9046f6d8825d48ee9f249557a32d1e38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } health_report { overall_health: HEALTHY } } }
I20260619 02:50:26.859864 21214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:26.854425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53848 (local address 127.18.134.3:42225)
0619 02:50:26.854567 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:26.854571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:26.854706 (+ 135us) server_negotiation.cc:407] Connection header received
0619 02:50:26.854930 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:26.854934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:26.854992 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:26.855055 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:50:26.855430 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.856118 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.858044 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.858497 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.858613 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:26.859101 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:50:26.859213 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:50:26.859311 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:26.859494 (+ 183us) server_negotiation.cc:299] Negotiation successful
0619 02:50:26.859589 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 02:50:26.877877 21264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:26.873464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:42225 (local address 127.18.134.3:59259)
0619 02:50:26.873668 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:26.873685 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:26.873833 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:26.874087 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:26.874092 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:26.874101 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:50:26.874296 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:26.874305 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.875343 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.875349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:26.876288 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:26.876299 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.876909 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.876915 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:26.877030 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:26.877062 (+ 32us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:50:26.877382 (+ 320us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:50:26.877392 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 02:50:26.877463 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 02:50:26.877587 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":18}
I20260619 02:50:26.877877 21214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:26.873523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:59259 (local address 127.18.134.3:42225)
0619 02:50:26.873664 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:26.873670 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:26.873747 (+ 77us) server_negotiation.cc:407] Connection header received
0619 02:50:26.873937 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:26.873942 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:26.874005 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:26.874070 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:50:26.874402 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.875215 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.876409 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:26.876782 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:26.876879 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:26.877190 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:50:26.877300 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:50:26.877369 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:26.877484 (+ 115us) server_negotiation.cc:299] Negotiation successful
0619 02:50:26.877547 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":52}
I20260619 02:50:26.887446 20870 raft_consensus.cc:1261] T 8a5ec0be444d4d8fa5cfef871812cc4a P e1412605cc804b8eb1a5a66a30889725 [term 1 FOLLOWER]: Refusing update from remote peer 9046f6d8825d48ee9f249557a32d1e38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:50:26.887449 21008 raft_consensus.cc:1261] T 8a5ec0be444d4d8fa5cfef871812cc4a P e2beb7aca7cf48289f69387c69077692 [term 1 FOLLOWER]: Refusing update from remote peer 9046f6d8825d48ee9f249557a32d1e38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:50:26.888051 21259 consensus_queue.cc:1035] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:50:26.888602 21275 consensus_queue.cc:1035] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260619 02:50:26.912372 21262 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 02:50:28.498234 20713 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:50:28.498447 20713 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:50:28.502292 20713 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6a59ebf29ad4eeab57577c61122f952: Sending DeleteTablet for 3 replicas of tablet 8a5ec0be444d4d8fa5cfef871812cc4a
I20260619 02:50:28.503091 20849 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a5ec0be444d4d8fa5cfef871812cc4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:50:28 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:35976
I20260619 02:50:28.503150 20988 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a5ec0be444d4d8fa5cfef871812cc4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:50:28 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:34522
I20260619 02:50:28.503139 21127 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a5ec0be444d4d8fa5cfef871812cc4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:50:28 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:53818
I20260619 02:50:28.515879 21301 tablet_replica.cc:330] T 8a5ec0be444d4d8fa5cfef871812cc4a P e2beb7aca7cf48289f69387c69077692: stopping tablet replica
I20260619 02:50:28.516103 21301 raft_consensus.cc:2229] T 8a5ec0be444d4d8fa5cfef871812cc4a P e2beb7aca7cf48289f69387c69077692 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:50:28.516157 21302 tablet_replica.cc:330] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38: stopping tablet replica
I20260619 02:50:28.516280 21300 tablet_replica.cc:330] T 8a5ec0be444d4d8fa5cfef871812cc4a P e1412605cc804b8eb1a5a66a30889725: stopping tablet replica
I20260619 02:50:28.516366 21301 raft_consensus.cc:2258] T 8a5ec0be444d4d8fa5cfef871812cc4a P e2beb7aca7cf48289f69387c69077692 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:50:28.516472 21300 raft_consensus.cc:2229] T 8a5ec0be444d4d8fa5cfef871812cc4a P e1412605cc804b8eb1a5a66a30889725 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:50:28.516533 21302 raft_consensus.cc:2229] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:50:28.516710 21300 raft_consensus.cc:2258] T 8a5ec0be444d4d8fa5cfef871812cc4a P e1412605cc804b8eb1a5a66a30889725 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:50:28.517215 21302 raft_consensus.cc:2258] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:50:28.517488 21301 ts_tablet_manager.cc:1905] T 8a5ec0be444d4d8fa5cfef871812cc4a P e2beb7aca7cf48289f69387c69077692: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:50:28.517861 21300 ts_tablet_manager.cc:1905] T 8a5ec0be444d4d8fa5cfef871812cc4a P e1412605cc804b8eb1a5a66a30889725: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:50:28.518147 21302 ts_tablet_manager.cc:1905] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:50:28.523758 21301 ts_tablet_manager.cc:1918] T 8a5ec0be444d4d8fa5cfef871812cc4a P e2beb7aca7cf48289f69387c69077692: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 02:50:28.523893 21301 log.cc:1196] T 8a5ec0be444d4d8fa5cfef871812cc4a P e2beb7aca7cf48289f69387c69077692: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-1/wal/wals/8a5ec0be444d4d8fa5cfef871812cc4a
I20260619 02:50:28.524016 21300 ts_tablet_manager.cc:1918] T 8a5ec0be444d4d8fa5cfef871812cc4a P e1412605cc804b8eb1a5a66a30889725: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 02:50:28.524156 21300 log.cc:1196] T 8a5ec0be444d4d8fa5cfef871812cc4a P e1412605cc804b8eb1a5a66a30889725: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-0/wal/wals/8a5ec0be444d4d8fa5cfef871812cc4a
I20260619 02:50:28.524281 21301 ts_tablet_manager.cc:1939] T 8a5ec0be444d4d8fa5cfef871812cc4a P e2beb7aca7cf48289f69387c69077692: Deleting consensus metadata
I20260619 02:50:28.524590 21300 ts_tablet_manager.cc:1939] T 8a5ec0be444d4d8fa5cfef871812cc4a P e1412605cc804b8eb1a5a66a30889725: Deleting consensus metadata
I20260619 02:50:28.524849 21302 ts_tablet_manager.cc:1918] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 02:50:28.525012 21302 log.cc:1196] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-2/wal/wals/8a5ec0be444d4d8fa5cfef871812cc4a
I20260619 02:50:28.525381 20697 catalog_manager.cc:4841] TS e2beb7aca7cf48289f69387c69077692 (127.18.134.2:42985): tablet 8a5ec0be444d4d8fa5cfef871812cc4a (table test-table [id=02227f7c12234fd08c5d5a87a1879da6]) successfully deleted
I20260619 02:50:28.525592 20698 catalog_manager.cc:4841] TS e1412605cc804b8eb1a5a66a30889725 (127.18.134.1:38701): tablet 8a5ec0be444d4d8fa5cfef871812cc4a (table test-table [id=02227f7c12234fd08c5d5a87a1879da6]) successfully deleted
I20260619 02:50:28.525683 21302 ts_tablet_manager.cc:1939] T 8a5ec0be444d4d8fa5cfef871812cc4a P 9046f6d8825d48ee9f249557a32d1e38: Deleting consensus metadata
I20260619 02:50:28.526512 20699 catalog_manager.cc:4841] TS 9046f6d8825d48ee9f249557a32d1e38 (127.18.134.3:42225): tablet 8a5ec0be444d4d8fa5cfef871812cc4a (table test-table [id=02227f7c12234fd08c5d5a87a1879da6]) successfully deleted
Jun 19 02:50:28 dist-test-slave-9mr0 krb5kdc[20668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837428, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 02:50:28 dist-test-slave-9mr0 krb5kdc[20668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837428, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:28.556352 21310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:28.542837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58772 (local address 127.18.134.62:35529)
0619 02:50:28.543405 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:28.543411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:28.543504 (+ 93us) server_negotiation.cc:407] Connection header received
0619 02:50:28.543699 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:28.543703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:28.543765 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:28.543855 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:28.545136 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:28.545911 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:28.547117 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:28.547526 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:28.547633 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:28.551590 (+ 3957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:28.551596 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:28.551599 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:28.551631 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:28.553939 (+ 2308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:28.554590 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:28.554595 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:28.554597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:28.554758 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:28.555104 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:28.555108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:28.555110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:28.555363 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:28.555527 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:28.555834 (+ 307us) server_negotiation.cc:299] Negotiation successful
0619 02:50:28.556005 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":264,"threads_started":1}
I20260619 02:50:28.559844 20713 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58772:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:50:28.560496 20713 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:50:28.566646 20849 tablet_service.cc:1467] Processing CreateTablet for tablet 6702ed6ac8d441f782fec6668c4ce9a2 (DEFAULT_TABLE table=test-table [id=3998e67bea69473983a876ccd238d61b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:50:28.566896 20988 tablet_service.cc:1467] Processing CreateTablet for tablet 6702ed6ac8d441f782fec6668c4ce9a2 (DEFAULT_TABLE table=test-table [id=3998e67bea69473983a876ccd238d61b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:50:28.566897 21127 tablet_service.cc:1467] Processing CreateTablet for tablet 6702ed6ac8d441f782fec6668c4ce9a2 (DEFAULT_TABLE table=test-table [id=3998e67bea69473983a876ccd238d61b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:50:28.567108 20849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6702ed6ac8d441f782fec6668c4ce9a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:28.567334 21127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6702ed6ac8d441f782fec6668c4ce9a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:28.567337 20988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6702ed6ac8d441f782fec6668c4ce9a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:28.572158 21311 tablet_bootstrap.cc:492] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725: Bootstrap starting.
I20260619 02:50:28.573467 21311 tablet_bootstrap.cc:654] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:28.574000 21312 tablet_bootstrap.cc:492] T 6702ed6ac8d441f782fec6668c4ce9a2 P 9046f6d8825d48ee9f249557a32d1e38: Bootstrap starting.
I20260619 02:50:28.574524 21313 tablet_bootstrap.cc:492] T 6702ed6ac8d441f782fec6668c4ce9a2 P e2beb7aca7cf48289f69387c69077692: Bootstrap starting.
I20260619 02:50:28.576066 21311 tablet_bootstrap.cc:492] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725: No bootstrap required, opened a new log
I20260619 02:50:28.576201 21311 ts_tablet_manager.cc:1397] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260619 02:50:28.576264 21312 tablet_bootstrap.cc:654] T 6702ed6ac8d441f782fec6668c4ce9a2 P 9046f6d8825d48ee9f249557a32d1e38: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:28.576586 21313 tablet_bootstrap.cc:654] T 6702ed6ac8d441f782fec6668c4ce9a2 P e2beb7aca7cf48289f69387c69077692: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:28.576835 21311 raft_consensus.cc:348] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:28.577353 21311 raft_consensus.cc:374] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:28.578056 21311 raft_consensus.cc:729] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1412605cc804b8eb1a5a66a30889725, State: Initialized, Role: FOLLOWER
I20260619 02:50:28.578327 21311 consensus_queue.cc:260] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:28.579018 21313 tablet_bootstrap.cc:492] T 6702ed6ac8d441f782fec6668c4ce9a2 P e2beb7aca7cf48289f69387c69077692: No bootstrap required, opened a new log
I20260619 02:50:28.579149 21311 ts_tablet_manager.cc:1428] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260619 02:50:28.579216 21313 ts_tablet_manager.cc:1397] T 6702ed6ac8d441f782fec6668c4ce9a2 P e2beb7aca7cf48289f69387c69077692: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260619 02:50:28.578962 21312 tablet_bootstrap.cc:492] T 6702ed6ac8d441f782fec6668c4ce9a2 P 9046f6d8825d48ee9f249557a32d1e38: No bootstrap required, opened a new log
I20260619 02:50:28.579366 21312 ts_tablet_manager.cc:1397] T 6702ed6ac8d441f782fec6668c4ce9a2 P 9046f6d8825d48ee9f249557a32d1e38: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260619 02:50:28.579965 21313 raft_consensus.cc:348] T 6702ed6ac8d441f782fec6668c4ce9a2 P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:28.580124 21312 raft_consensus.cc:348] T 6702ed6ac8d441f782fec6668c4ce9a2 P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:28.580279 21313 raft_consensus.cc:374] T 6702ed6ac8d441f782fec6668c4ce9a2 P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:28.580355 21312 raft_consensus.cc:374] T 6702ed6ac8d441f782fec6668c4ce9a2 P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:28.580379 21313 raft_consensus.cc:729] T 6702ed6ac8d441f782fec6668c4ce9a2 P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2beb7aca7cf48289f69387c69077692, State: Initialized, Role: FOLLOWER
I20260619 02:50:28.580456 21312 raft_consensus.cc:729] T 6702ed6ac8d441f782fec6668c4ce9a2 P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9046f6d8825d48ee9f249557a32d1e38, State: Initialized, Role: FOLLOWER
I20260619 02:50:28.580669 21312 consensus_queue.cc:260] T 6702ed6ac8d441f782fec6668c4ce9a2 P 9046f6d8825d48ee9f249557a32d1e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:28.580662 21313 consensus_queue.cc:260] T 6702ed6ac8d441f782fec6668c4ce9a2 P e2beb7aca7cf48289f69387c69077692 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:28.581331 21312 ts_tablet_manager.cc:1428] T 6702ed6ac8d441f782fec6668c4ce9a2 P 9046f6d8825d48ee9f249557a32d1e38: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:50:28.582424 21313 ts_tablet_manager.cc:1428] T 6702ed6ac8d441f782fec6668c4ce9a2 P e2beb7aca7cf48289f69387c69077692: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 02:50:28.718133 21317 raft_consensus.cc:482] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:50:28.718384 21317 raft_consensus.cc:504] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:28.719942 21317 leader_election.cc:288] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2beb7aca7cf48289f69387c69077692 (127.18.134.2:42985), 9046f6d8825d48ee9f249557a32d1e38 (127.18.134.3:42225)
I20260619 02:50:28.736413 21322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:28.720372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:42761 (local address 127.18.134.3:42225)
0619 02:50:28.730726 (+ 10354us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:28.730742 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:28.731241 (+ 499us) server_negotiation.cc:407] Connection header received
0619 02:50:28.731410 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:28.731416 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:28.731486 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:28.731559 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:28.732231 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:28.732999 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:28.734678 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:28.735441 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:28.735599 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:28.735669 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:28.735973 (+ 304us) server_negotiation.cc:299] Negotiation successful
0619 02:50:28.736103 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10199,"thread_start_us":9604,"threads_started":1}
I20260619 02:50:28.736423 21321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:28.720325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:60139 (local address 127.18.134.2:42985)
0619 02:50:28.730804 (+ 10479us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:28.730810 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:28.731372 (+ 562us) server_negotiation.cc:407] Connection header received
0619 02:50:28.731518 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:28.731522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:28.731589 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:28.731693 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:28.732265 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:28.732982 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:28.734693 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:28.735436 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:28.735581 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:28.735657 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:28.735958 (+ 301us) server_negotiation.cc:299] Negotiation successful
0619 02:50:28.736098 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10352,"thread_start_us":10202,"threads_started":1}
I20260619 02:50:28.736598 21320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:28.720214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:42985 (local address 127.18.134.1:60139)
0619 02:50:28.731248 (+ 11034us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:28.731270 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:28.731400 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:28.731725 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:28.731730 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:28.731773 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:28.732096 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:28.732107 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:28.733100 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:28.733105 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:28.734527 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:28.734538 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:28.735594 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:28.735600 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:28.735759 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:28.735836 (+ 77us) client_negotiation.cc:769] Sending connection context
0619 02:50:28.735910 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 02:50:28.736104 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10912,"mutex_wait_us":169,"spinlock_wait_cycles":1280,"thread_start_us":10645,"threads_started":1}
I20260619 02:50:28.736910 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:28.720214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:42225 (local address 127.18.134.1:42761)
0619 02:50:28.731122 (+ 10908us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:28.731141 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:28.731277 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:28.731602 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:28.731606 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:28.731631 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:28.731980 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:28.731993 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:28.733135 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:28.733139 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:28.734522 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:28.734533 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:28.735632 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:28.735637 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:28.735808 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:28.735846 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 02:50:28.735947 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 02:50:28.736132 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10760,"mutex_wait_us":9,"spinlock_wait_cycles":28672,"thread_start_us":10578,"threads_started":1}
I20260619 02:50:28.737190 21008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6702ed6ac8d441f782fec6668c4ce9a2" candidate_uuid: "e1412605cc804b8eb1a5a66a30889725" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2beb7aca7cf48289f69387c69077692" is_pre_election: true
I20260619 02:50:28.737498 21008 raft_consensus.cc:2454] T 6702ed6ac8d441f782fec6668c4ce9a2 P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1412605cc804b8eb1a5a66a30889725 in term 0.
I20260619 02:50:28.737510 21147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6702ed6ac8d441f782fec6668c4ce9a2" candidate_uuid: "e1412605cc804b8eb1a5a66a30889725" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9046f6d8825d48ee9f249557a32d1e38" is_pre_election: true
I20260619 02:50:28.738103 20800 leader_election.cc:302] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e1412605cc804b8eb1a5a66a30889725, e2beb7aca7cf48289f69387c69077692; no voters:
I20260619 02:50:28.738226 21147 raft_consensus.cc:2454] T 6702ed6ac8d441f782fec6668c4ce9a2 P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1412605cc804b8eb1a5a66a30889725 in term 0.
I20260619 02:50:28.738511 21317 raft_consensus.cc:2781] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:50:28.738621 21317 raft_consensus.cc:482] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:50:28.738749 21317 raft_consensus.cc:3037] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:28.741019 21317 raft_consensus.cc:504] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:28.741555 21317 leader_election.cc:288] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725 [CANDIDATE]: Term 1 election: Requested vote from peers e2beb7aca7cf48289f69387c69077692 (127.18.134.2:42985), 9046f6d8825d48ee9f249557a32d1e38 (127.18.134.3:42225)
I20260619 02:50:28.742036 21147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6702ed6ac8d441f782fec6668c4ce9a2" candidate_uuid: "e1412605cc804b8eb1a5a66a30889725" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9046f6d8825d48ee9f249557a32d1e38"
I20260619 02:50:28.742216 21147 raft_consensus.cc:3037] T 6702ed6ac8d441f782fec6668c4ce9a2 P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:28.742369 21008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6702ed6ac8d441f782fec6668c4ce9a2" candidate_uuid: "e1412605cc804b8eb1a5a66a30889725" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2beb7aca7cf48289f69387c69077692"
I20260619 02:50:28.742573 21008 raft_consensus.cc:3037] T 6702ed6ac8d441f782fec6668c4ce9a2 P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:28.745223 21147 raft_consensus.cc:2454] T 6702ed6ac8d441f782fec6668c4ce9a2 P 9046f6d8825d48ee9f249557a32d1e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1412605cc804b8eb1a5a66a30889725 in term 1.
I20260619 02:50:28.745222 21008 raft_consensus.cc:2454] T 6702ed6ac8d441f782fec6668c4ce9a2 P e2beb7aca7cf48289f69387c69077692 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1412605cc804b8eb1a5a66a30889725 in term 1.
I20260619 02:50:28.745601 20802 leader_election.cc:302] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9046f6d8825d48ee9f249557a32d1e38, e1412605cc804b8eb1a5a66a30889725; no voters:
I20260619 02:50:28.745919 21317 raft_consensus.cc:2781] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:50:28.746359 21317 raft_consensus.cc:686] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725 [term 1 LEADER]: Becoming Leader. State: Replica: e1412605cc804b8eb1a5a66a30889725, State: Running, Role: LEADER
I20260619 02:50:28.746779 21317 consensus_queue.cc:237] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:28.750456 20712 catalog_manager.cc:5495] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1412605cc804b8eb1a5a66a30889725 (127.18.134.1). New cstate: current_term: 1 leader_uuid: "e1412605cc804b8eb1a5a66a30889725" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } health_report { overall_health: UNKNOWN } } }
I20260619 02:50:28.783167 21327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:28.776275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35986 (local address 127.18.134.1:38701)
0619 02:50:28.776860 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:28.776864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:28.776882 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:50:28.776957 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:28.776960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:28.777025 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:28.777154 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:50:28.777960 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:28.778843 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:28.780719 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:28.781284 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:28.781384 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:28.781850 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:50:28.782486 (+ 636us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:50:28.782593 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:28.782724 (+ 131us) server_negotiation.cc:299] Negotiation successful
0619 02:50:28.782799 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":365,"threads_started":1}
I20260619 02:50:28.797703 21322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:28.792767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:41745 (local address 127.18.134.3:42225)
0619 02:50:28.792957 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:28.792962 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:28.793125 (+ 163us) server_negotiation.cc:407] Connection header received
0619 02:50:28.793257 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:28.793261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:28.793356 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:28.793427 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:50:28.794311 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:28.795044 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:28.796352 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:28.796731 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:28.796823 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:28.797175 (+ 352us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:50:28.797272 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:50:28.797367 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:28.797453 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 02:50:28.797512 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 02:50:28.797780 21330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:28.792461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:42225 (local address 127.18.134.1:41745)
0619 02:50:28.793039 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:28.793054 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:28.793165 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:28.793451 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:28.793454 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:28.793463 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:50:28.794145 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:28.794154 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:28.795177 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:28.795182 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:28.796235 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:28.796246 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:28.796859 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:28.796863 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:28.796997 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:28.797049 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:50:28.797372 (+ 323us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:50:28.797379 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 02:50:28.797433 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 02:50:28.797524 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":274,"threads_started":1}
I20260619 02:50:28.806069 21008 raft_consensus.cc:1261] T 6702ed6ac8d441f782fec6668c4ce9a2 P e2beb7aca7cf48289f69387c69077692 [term 1 FOLLOWER]: Refusing update from remote peer e1412605cc804b8eb1a5a66a30889725: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:50:28.806139 21147 raft_consensus.cc:1261] T 6702ed6ac8d441f782fec6668c4ce9a2 P 9046f6d8825d48ee9f249557a32d1e38 [term 1 FOLLOWER]: Refusing update from remote peer e1412605cc804b8eb1a5a66a30889725: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:50:28.806733 21323 consensus_queue.cc:1035] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:50:28.807060 21317 consensus_queue.cc:1035] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:50:28.833388 21327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:28.827963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:53935 (local address 127.18.134.1:38701)
0619 02:50:28.828127 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:28.828133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:28.828706 (+ 573us) server_negotiation.cc:407] Connection header received
0619 02:50:28.828822 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:28.828825 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:28.828882 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:28.828949 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:50:28.829407 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:28.830152 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:28.831817 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:28.832313 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:28.832410 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:28.832781 (+ 371us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:50:28.832873 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:50:28.832935 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:28.833074 (+ 139us) server_negotiation.cc:299] Negotiation successful
0619 02:50:28.833131 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 02:50:28.833441 21334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:28.827973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:38701 (local address 127.18.134.3:53935)
0619 02:50:28.828620 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:28.828635 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:28.828739 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:28.828963 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:28.828966 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:28.828980 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:50:28.829260 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:28.829272 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:28.830273 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:28.830278 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:28.831685 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:28.831698 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:28.832487 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:28.832492 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:28.832614 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:28.832659 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:50:28.832970 (+ 311us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:50:28.832981 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 02:50:28.833059 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 02:50:28.833155 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":366,"threads_started":1}
I20260619 02:50:28.864095 20711 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:50:28.864387 20711 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:50:28.867715 20711 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6a59ebf29ad4eeab57577c61122f952: Sending DeleteTablet for 3 replicas of tablet 6702ed6ac8d441f782fec6668c4ce9a2
I20260619 02:50:28.868481 20849 tablet_service.cc:1514] Processing DeleteTablet for tablet 6702ed6ac8d441f782fec6668c4ce9a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:50:28 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:35976
I20260619 02:50:28.868523 20988 tablet_service.cc:1514] Processing DeleteTablet for tablet 6702ed6ac8d441f782fec6668c4ce9a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:50:28 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:34522
I20260619 02:50:28.868664 21127 tablet_service.cc:1514] Processing DeleteTablet for tablet 6702ed6ac8d441f782fec6668c4ce9a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:50:28 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:53818
I20260619 02:50:28.868777 21300 tablet_replica.cc:330] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725: stopping tablet replica
I20260619 02:50:28.868782 21301 tablet_replica.cc:330] T 6702ed6ac8d441f782fec6668c4ce9a2 P e2beb7aca7cf48289f69387c69077692: stopping tablet replica
I20260619 02:50:28.868916 21302 tablet_replica.cc:330] T 6702ed6ac8d441f782fec6668c4ce9a2 P 9046f6d8825d48ee9f249557a32d1e38: stopping tablet replica
I20260619 02:50:28.868965 21301 raft_consensus.cc:2229] T 6702ed6ac8d441f782fec6668c4ce9a2 P e2beb7aca7cf48289f69387c69077692 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:50:28.868988 21300 raft_consensus.cc:2229] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:50:28.869078 21302 raft_consensus.cc:2229] T 6702ed6ac8d441f782fec6668c4ce9a2 P 9046f6d8825d48ee9f249557a32d1e38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:50:28.869139 21301 raft_consensus.cc:2258] T 6702ed6ac8d441f782fec6668c4ce9a2 P e2beb7aca7cf48289f69387c69077692 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:50:28.869232 21302 raft_consensus.cc:2258] T 6702ed6ac8d441f782fec6668c4ce9a2 P 9046f6d8825d48ee9f249557a32d1e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:50:28.869498 21300 raft_consensus.cc:2258] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:50:28.870294 21302 ts_tablet_manager.cc:1905] T 6702ed6ac8d441f782fec6668c4ce9a2 P 9046f6d8825d48ee9f249557a32d1e38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:50:28.870622 21300 ts_tablet_manager.cc:1905] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:50:28.875449 21301 ts_tablet_manager.cc:1905] T 6702ed6ac8d441f782fec6668c4ce9a2 P e2beb7aca7cf48289f69387c69077692: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:50:28.876173 20711 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58772:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:50:28.876816 20711 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:50:28.877429 21300 ts_tablet_manager.cc:1918] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:50:28.877607 21300 log.cc:1196] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-0/wal/wals/6702ed6ac8d441f782fec6668c4ce9a2
I20260619 02:50:28.878062 21300 ts_tablet_manager.cc:1939] T 6702ed6ac8d441f782fec6668c4ce9a2 P e1412605cc804b8eb1a5a66a30889725: Deleting consensus metadata
I20260619 02:50:28.878913 21302 ts_tablet_manager.cc:1918] T 6702ed6ac8d441f782fec6668c4ce9a2 P 9046f6d8825d48ee9f249557a32d1e38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:50:28.879142 21302 log.cc:1196] T 6702ed6ac8d441f782fec6668c4ce9a2 P 9046f6d8825d48ee9f249557a32d1e38: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-2/wal/wals/6702ed6ac8d441f782fec6668c4ce9a2
I20260619 02:50:28.879243 20698 catalog_manager.cc:4841] TS e1412605cc804b8eb1a5a66a30889725 (127.18.134.1:38701): tablet 6702ed6ac8d441f782fec6668c4ce9a2 (table test-table [id=3998e67bea69473983a876ccd238d61b]) successfully deleted
I20260619 02:50:28.879582 21302 ts_tablet_manager.cc:1939] T 6702ed6ac8d441f782fec6668c4ce9a2 P 9046f6d8825d48ee9f249557a32d1e38: Deleting consensus metadata
I20260619 02:50:28.880391 20699 catalog_manager.cc:4841] TS 9046f6d8825d48ee9f249557a32d1e38 (127.18.134.3:42225): tablet 6702ed6ac8d441f782fec6668c4ce9a2 (table test-table [id=3998e67bea69473983a876ccd238d61b]) successfully deleted
I20260619 02:50:28.883402 21301 ts_tablet_manager.cc:1918] T 6702ed6ac8d441f782fec6668c4ce9a2 P e2beb7aca7cf48289f69387c69077692: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:50:28.883556 21301 log.cc:1196] T 6702ed6ac8d441f782fec6668c4ce9a2 P e2beb7aca7cf48289f69387c69077692: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-1/wal/wals/6702ed6ac8d441f782fec6668c4ce9a2
I20260619 02:50:28.883949 21301 ts_tablet_manager.cc:1939] T 6702ed6ac8d441f782fec6668c4ce9a2 P e2beb7aca7cf48289f69387c69077692: Deleting consensus metadata
I20260619 02:50:28.884891 21127 tablet_service.cc:1467] Processing CreateTablet for tablet 28f72515353e40099d11fc6662fc5d79 (DEFAULT_TABLE table=test-table [id=3df2a748705148f6855779f34179ed6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:50:28.884905 20988 tablet_service.cc:1467] Processing CreateTablet for tablet 28f72515353e40099d11fc6662fc5d79 (DEFAULT_TABLE table=test-table [id=3df2a748705148f6855779f34179ed6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:50:28.885268 21127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28f72515353e40099d11fc6662fc5d79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:28.885280 20988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28f72515353e40099d11fc6662fc5d79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:28.886197 20849 tablet_service.cc:1467] Processing CreateTablet for tablet 28f72515353e40099d11fc6662fc5d79 (DEFAULT_TABLE table=test-table [id=3df2a748705148f6855779f34179ed6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:50:28.886601 20849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28f72515353e40099d11fc6662fc5d79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:28.890875 20697 catalog_manager.cc:4841] TS e2beb7aca7cf48289f69387c69077692 (127.18.134.2:42985): tablet 6702ed6ac8d441f782fec6668c4ce9a2 (table test-table [id=3998e67bea69473983a876ccd238d61b]) successfully deleted
I20260619 02:50:28.891165 21312 tablet_bootstrap.cc:492] T 28f72515353e40099d11fc6662fc5d79 P 9046f6d8825d48ee9f249557a32d1e38: Bootstrap starting.
I20260619 02:50:28.891736 21313 tablet_bootstrap.cc:492] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692: Bootstrap starting.
I20260619 02:50:28.892637 21312 tablet_bootstrap.cc:654] T 28f72515353e40099d11fc6662fc5d79 P 9046f6d8825d48ee9f249557a32d1e38: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:28.893190 21313 tablet_bootstrap.cc:654] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:28.895187 21311 tablet_bootstrap.cc:492] T 28f72515353e40099d11fc6662fc5d79 P e1412605cc804b8eb1a5a66a30889725: Bootstrap starting.
I20260619 02:50:28.896768 21311 tablet_bootstrap.cc:654] T 28f72515353e40099d11fc6662fc5d79 P e1412605cc804b8eb1a5a66a30889725: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:28.902851 21311 tablet_bootstrap.cc:492] T 28f72515353e40099d11fc6662fc5d79 P e1412605cc804b8eb1a5a66a30889725: No bootstrap required, opened a new log
I20260619 02:50:28.902981 21313 tablet_bootstrap.cc:492] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692: No bootstrap required, opened a new log
I20260619 02:50:28.903044 21311 ts_tablet_manager.cc:1397] T 28f72515353e40099d11fc6662fc5d79 P e1412605cc804b8eb1a5a66a30889725: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.006s
I20260619 02:50:28.903136 21313 ts_tablet_manager.cc:1397] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.008s
I20260619 02:50:28.903440 21312 tablet_bootstrap.cc:492] T 28f72515353e40099d11fc6662fc5d79 P 9046f6d8825d48ee9f249557a32d1e38: No bootstrap required, opened a new log
I20260619 02:50:28.903613 21312 ts_tablet_manager.cc:1397] T 28f72515353e40099d11fc6662fc5d79 P 9046f6d8825d48ee9f249557a32d1e38: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260619 02:50:28.903949 21313 raft_consensus.cc:348] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:28.904282 21312 raft_consensus.cc:348] T 28f72515353e40099d11fc6662fc5d79 P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:28.904501 21312 raft_consensus.cc:374] T 28f72515353e40099d11fc6662fc5d79 P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:28.904174 21313 raft_consensus.cc:374] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:28.904392 21311 raft_consensus.cc:348] T 28f72515353e40099d11fc6662fc5d79 P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:28.904564 21312 raft_consensus.cc:729] T 28f72515353e40099d11fc6662fc5d79 P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9046f6d8825d48ee9f249557a32d1e38, State: Initialized, Role: FOLLOWER
I20260619 02:50:28.904599 21313 raft_consensus.cc:729] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2beb7aca7cf48289f69387c69077692, State: Initialized, Role: FOLLOWER
I20260619 02:50:28.904649 21311 raft_consensus.cc:374] T 28f72515353e40099d11fc6662fc5d79 P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:28.904732 21311 raft_consensus.cc:729] T 28f72515353e40099d11fc6662fc5d79 P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1412605cc804b8eb1a5a66a30889725, State: Initialized, Role: FOLLOWER
I20260619 02:50:28.904837 21313 consensus_queue.cc:260] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:28.905470 21313 ts_tablet_manager.cc:1428] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:50:28.905916 21312 consensus_queue.cc:260] T 28f72515353e40099d11fc6662fc5d79 P 9046f6d8825d48ee9f249557a32d1e38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:28.904992 21311 consensus_queue.cc:260] T 28f72515353e40099d11fc6662fc5d79 P e1412605cc804b8eb1a5a66a30889725 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:28.906418 21312 ts_tablet_manager.cc:1428] T 28f72515353e40099d11fc6662fc5d79 P 9046f6d8825d48ee9f249557a32d1e38: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260619 02:50:28.909451 21311 ts_tablet_manager.cc:1428] T 28f72515353e40099d11fc6662fc5d79 P e1412605cc804b8eb1a5a66a30889725: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260619 02:50:28.926707 21318 raft_consensus.cc:482] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:50:28.926959 21318 raft_consensus.cc:504] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:28.927820 21318 leader_election.cc:288] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1412605cc804b8eb1a5a66a30889725 (127.18.134.1:38701), 9046f6d8825d48ee9f249557a32d1e38 (127.18.134.3:42225)
I20260619 02:50:28.928261 20870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28f72515353e40099d11fc6662fc5d79" candidate_uuid: "e2beb7aca7cf48289f69387c69077692" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1412605cc804b8eb1a5a66a30889725" is_pre_election: true
I20260619 02:50:28.928321 21147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28f72515353e40099d11fc6662fc5d79" candidate_uuid: "e2beb7aca7cf48289f69387c69077692" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9046f6d8825d48ee9f249557a32d1e38" is_pre_election: true
I20260619 02:50:28.928486 20870 raft_consensus.cc:2454] T 28f72515353e40099d11fc6662fc5d79 P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2beb7aca7cf48289f69387c69077692 in term 0.
I20260619 02:50:28.928583 21147 raft_consensus.cc:2454] T 28f72515353e40099d11fc6662fc5d79 P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2beb7aca7cf48289f69387c69077692 in term 0.
I20260619 02:50:28.928968 20941 leader_election.cc:302] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e1412605cc804b8eb1a5a66a30889725, e2beb7aca7cf48289f69387c69077692; no voters:
I20260619 02:50:28.929291 21318 raft_consensus.cc:2781] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:50:28.929425 21318 raft_consensus.cc:482] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:50:28.929522 21318 raft_consensus.cc:3037] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:28.932552 21318 raft_consensus.cc:504] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:28.933655 20870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28f72515353e40099d11fc6662fc5d79" candidate_uuid: "e2beb7aca7cf48289f69387c69077692" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1412605cc804b8eb1a5a66a30889725"
I20260619 02:50:28.933755 21147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28f72515353e40099d11fc6662fc5d79" candidate_uuid: "e2beb7aca7cf48289f69387c69077692" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9046f6d8825d48ee9f249557a32d1e38"
I20260619 02:50:28.933874 20870 raft_consensus.cc:3037] T 28f72515353e40099d11fc6662fc5d79 P e1412605cc804b8eb1a5a66a30889725 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:28.933986 21147 raft_consensus.cc:3037] T 28f72515353e40099d11fc6662fc5d79 P 9046f6d8825d48ee9f249557a32d1e38 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:28.936043 20870 raft_consensus.cc:2454] T 28f72515353e40099d11fc6662fc5d79 P e1412605cc804b8eb1a5a66a30889725 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2beb7aca7cf48289f69387c69077692 in term 1.
I20260619 02:50:28.936240 21318 leader_election.cc:288] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692 [CANDIDATE]: Term 1 election: Requested vote from peers e1412605cc804b8eb1a5a66a30889725 (127.18.134.1:38701), 9046f6d8825d48ee9f249557a32d1e38 (127.18.134.3:42225)
I20260619 02:50:28.936491 20941 leader_election.cc:302] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e1412605cc804b8eb1a5a66a30889725, e2beb7aca7cf48289f69387c69077692; no voters:
I20260619 02:50:28.936714 21318 raft_consensus.cc:2781] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:50:28.937120 21318 raft_consensus.cc:686] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692 [term 1 LEADER]: Becoming Leader. State: Replica: e2beb7aca7cf48289f69387c69077692, State: Running, Role: LEADER
I20260619 02:50:28.937422 21147 raft_consensus.cc:2454] T 28f72515353e40099d11fc6662fc5d79 P 9046f6d8825d48ee9f249557a32d1e38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2beb7aca7cf48289f69387c69077692 in term 1.
I20260619 02:50:28.937413 21318 consensus_queue.cc:237] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } }
I20260619 02:50:28.939796 20713 catalog_manager.cc:5495] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2beb7aca7cf48289f69387c69077692 (127.18.134.2). New cstate: current_term: 1 leader_uuid: "e2beb7aca7cf48289f69387c69077692" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2beb7aca7cf48289f69387c69077692" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 } health_report { overall_health: UNKNOWN } } }
I20260619 02:50:28.960628 21321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:28.955864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34534 (local address 127.18.134.2:42985)
0619 02:50:28.956027 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:28.956033 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:28.956262 (+ 229us) server_negotiation.cc:407] Connection header received
0619 02:50:28.956436 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:28.956442 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:28.956513 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:28.956585 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:50:28.956950 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:28.957747 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:28.959087 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:28.959474 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:28.959578 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:28.959992 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:50:28.960096 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:50:28.960165 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:28.960299 (+ 134us) server_negotiation.cc:299] Negotiation successful
0619 02:50:28.960374 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 02:50:28.969543 21147 raft_consensus.cc:1261] T 28f72515353e40099d11fc6662fc5d79 P 9046f6d8825d48ee9f249557a32d1e38 [term 1 FOLLOWER]: Refusing update from remote peer e2beb7aca7cf48289f69387c69077692: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:50:28.969614 20870 raft_consensus.cc:1261] T 28f72515353e40099d11fc6662fc5d79 P e1412605cc804b8eb1a5a66a30889725 [term 1 FOLLOWER]: Refusing update from remote peer e2beb7aca7cf48289f69387c69077692: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:50:28.970158 21340 consensus_queue.cc:1035] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1412605cc804b8eb1a5a66a30889725" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 38701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:50:28.970469 21318 consensus_queue.cc:1035] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9046f6d8825d48ee9f249557a32d1e38" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 02:50:28.992022 21284 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 02:50:30.566051 20713 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:50:30.566264 20713 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:50:30.570417 20713 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6a59ebf29ad4eeab57577c61122f952: Sending DeleteTablet for 3 replicas of tablet 28f72515353e40099d11fc6662fc5d79
I20260619 02:50:30.571158 20849 tablet_service.cc:1514] Processing DeleteTablet for tablet 28f72515353e40099d11fc6662fc5d79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:50:30 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:35976
I20260619 02:50:30.571262 21127 tablet_service.cc:1514] Processing DeleteTablet for tablet 28f72515353e40099d11fc6662fc5d79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:50:30 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:53818
I20260619 02:50:30.571282 20988 tablet_service.cc:1514] Processing DeleteTablet for tablet 28f72515353e40099d11fc6662fc5d79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:50:30 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:34522
I20260619 02:50:30.572458 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 20784
I20260619 02:50:30.584698 21368 tablet_replica.cc:330] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692: stopping tablet replica
I20260619 02:50:30.584949 21368 raft_consensus.cc:2229] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:50:30.585366 21368 raft_consensus.cc:2258] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:50:30.586511 21368 ts_tablet_manager.cc:1905] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:50:30.588891 21366 tablet_replica.cc:330] T 28f72515353e40099d11fc6662fc5d79 P e1412605cc804b8eb1a5a66a30889725: stopping tablet replica
I20260619 02:50:30.589115 21366 raft_consensus.cc:2229] T 28f72515353e40099d11fc6662fc5d79 P e1412605cc804b8eb1a5a66a30889725 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:50:30.589362 21366 raft_consensus.cc:2258] T 28f72515353e40099d11fc6662fc5d79 P e1412605cc804b8eb1a5a66a30889725 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:50:30.589869 21367 tablet_replica.cc:330] T 28f72515353e40099d11fc6662fc5d79 P 9046f6d8825d48ee9f249557a32d1e38: stopping tablet replica
I20260619 02:50:30.590044 21367 raft_consensus.cc:2229] T 28f72515353e40099d11fc6662fc5d79 P 9046f6d8825d48ee9f249557a32d1e38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:50:30.590242 21367 raft_consensus.cc:2258] T 28f72515353e40099d11fc6662fc5d79 P 9046f6d8825d48ee9f249557a32d1e38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:50:30.591079 21366 ts_tablet_manager.cc:1905] T 28f72515353e40099d11fc6662fc5d79 P e1412605cc804b8eb1a5a66a30889725: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:50:30.591615 21367 ts_tablet_manager.cc:1905] T 28f72515353e40099d11fc6662fc5d79 P 9046f6d8825d48ee9f249557a32d1e38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:50:30.592957 21368 ts_tablet_manager.cc:1918] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 02:50:30.593109 21368 log.cc:1196] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-1/wal/wals/28f72515353e40099d11fc6662fc5d79
I20260619 02:50:30.593623 21368 ts_tablet_manager.cc:1939] T 28f72515353e40099d11fc6662fc5d79 P e2beb7aca7cf48289f69387c69077692: Deleting consensus metadata
I20260619 02:50:30.594746 20697 catalog_manager.cc:4841] TS e2beb7aca7cf48289f69387c69077692 (127.18.134.2:42985): tablet 28f72515353e40099d11fc6662fc5d79 (table test-table [id=3df2a748705148f6855779f34179ed6d]) successfully deleted
Jun 19 02:50:30 dist-test-slave-9mr0 krb5kdc[20668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837428, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.134.1@KRBTEST.COM
I20260619 02:50:30.597262 21366 ts_tablet_manager.cc:1918] T 28f72515353e40099d11fc6662fc5d79 P e1412605cc804b8eb1a5a66a30889725: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 02:50:30.597424 21366 log.cc:1196] T 28f72515353e40099d11fc6662fc5d79 P e1412605cc804b8eb1a5a66a30889725: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-0/wal/wals/28f72515353e40099d11fc6662fc5d79
I20260619 02:50:30.597858 21366 ts_tablet_manager.cc:1939] T 28f72515353e40099d11fc6662fc5d79 P e1412605cc804b8eb1a5a66a30889725: Deleting consensus metadata
I20260619 02:50:30.598443 21367 ts_tablet_manager.cc:1918] T 28f72515353e40099d11fc6662fc5d79 P 9046f6d8825d48ee9f249557a32d1e38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 02:50:30.598573 21367 log.cc:1196] T 28f72515353e40099d11fc6662fc5d79 P 9046f6d8825d48ee9f249557a32d1e38: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781837410515220-18968-0/minicluster-data/ts-2/wal/wals/28f72515353e40099d11fc6662fc5d79
I20260619 02:50:30.598919 20698 catalog_manager.cc:4841] TS e1412605cc804b8eb1a5a66a30889725 (127.18.134.1:38701): tablet 28f72515353e40099d11fc6662fc5d79 (table test-table [id=3df2a748705148f6855779f34179ed6d]) successfully deleted
I20260619 02:50:30.599021 21367 ts_tablet_manager.cc:1939] T 28f72515353e40099d11fc6662fc5d79 P 9046f6d8825d48ee9f249557a32d1e38: Deleting consensus metadata
I20260619 02:50:30.599850 20699 catalog_manager.cc:4841] TS 9046f6d8825d48ee9f249557a32d1e38 (127.18.134.3:42225): tablet 28f72515353e40099d11fc6662fc5d79 (table test-table [id=3df2a748705148f6855779f34179ed6d]) successfully deleted
I20260619 02:50:30.602751 21370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:30.574277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55184 (local address 127.18.134.1:38701)
0619 02:50:30.584970 (+ 10693us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:30.584978 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:30.585796 (+ 818us) server_negotiation.cc:407] Connection header received
0619 02:50:30.585877 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:30.585881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:30.585969 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:30.586066 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:30.587859 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:30.589031 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:30.591157 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:30.591824 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:30.591953 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:30.597058 (+ 5105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:30.597072 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:30.597089 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:30.597126 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:30.600141 (+ 3015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:30.600756 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:30.600763 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:30.600766 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:30.600894 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:30.601239 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:30.601242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:30.601244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:30.601593 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:30.601849 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:30.602150 (+ 301us) server_negotiation.cc:299] Negotiation successful
0619 02:50:30.602360 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10455,"thread_start_us":10326,"threads_started":1}
W20260619 02:50:30.603406 20915 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55184
W20260619 02:50:30.603982 18968 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 20784: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 02:50:30.604167 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 20784
I20260619 02:50:30.621492 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 20924
Jun 19 02:50:30 dist-test-slave-9mr0 krb5kdc[20668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837428, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.134.2@KRBTEST.COM
I20260619 02:50:30.635988 21371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:30.622208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55464 (local address 127.18.134.2:42985)
0619 02:50:30.622799 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:30.622804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:30.622819 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:50:30.622871 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:30.622875 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:30.622935 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:30.623021 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:30.624121 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:30.624892 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:30.626020 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:30.626436 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:30.626537 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:30.630733 (+ 4196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:30.630751 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:30.630768 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:30.630812 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:30.633376 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:30.633986 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:30.633993 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:30.633996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:30.634137 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:30.634555 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:30.634559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:30.634561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:30.634980 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:30.635191 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:30.635503 (+ 312us) server_negotiation.cc:299] Negotiation successful
0619 02:50:30.635672 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":280,"threads_started":1}
W20260619 02:50:30.636654 21054 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55464
W20260619 02:50:30.637264 18968 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 20924: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 02:50:30.637392 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 20924
I20260619 02:50:30.655974 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 21063
Jun 19 02:50:30 dist-test-slave-9mr0 krb5kdc[20668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837428, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.134.3@KRBTEST.COM
I20260619 02:50:30.670840 21372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:30.656607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58516 (local address 127.18.134.3:42225)
0619 02:50:30.657065 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:30.657069 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:30.657085 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:50:30.657134 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:30.657137 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:30.657197 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:30.657288 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:30.658541 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:30.659359 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:30.660751 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:30.661173 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:30.661285 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:30.665545 (+ 4260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:30.665561 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:30.665576 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:30.665613 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:30.668341 (+ 2728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:30.669021 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:30.669029 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:30.669031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:30.669175 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:30.669491 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:30.669494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:30.669496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:30.669839 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:30.670052 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:30.670259 (+ 207us) server_negotiation.cc:299] Negotiation successful
0619 02:50:30.670417 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":215,"threads_started":1}
W20260619 02:50:30.671542 21193 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58516
W20260619 02:50:30.672111 18968 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 21063: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 02:50:30.672264 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 21063
I20260619 02:50:30.691455 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 20681
I20260619 02:50:30.692693 20753 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:30.802709 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 20681
2026-06-19T02:50:30Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8226 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:50:30 dist-test-slave-9mr0 krb5kdc[21376](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:50:30 dist-test-slave-9mr0 krb5kdc[21376](info): set up 2 sockets
Jun 19 02:50:30 dist-test-slave-9mr0 krb5kdc[21376](info): commencing operation
krb5kdc: starting...
W20260619 02:50:31.798703 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.957s user 0.002s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:50:31 dist-test-slave-9mr0 krb5kdc[21376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:50:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:50:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:50:31.974231 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:45425
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:46433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:45425
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:32.283614 21389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:32.283939 21389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:32.284053 21389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:32.291360 21389 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:50:32.291486 21389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:32.291564 21389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:32.291635 21389 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:50:32.291695 21389 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:50:32.301672 21389 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:46433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:45425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:45425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:32.303012 21389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:32.304291 21389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:32.315934 21397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:32.315989 21394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:32.315945 21395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:32.316782 21389 server_base.cc:1034] running on GCE node
I20260619 02:50:32.317560 21389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:32.318979 21389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:32.320199 21389 hybrid_clock.cc:648] HybridClock initialized: now 1781837432320140 us; error 44 us; skew 500 ppm
I20260619 02:50:32.320544 21389 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:32 dist-test-slave-9mr0 krb5kdc[21376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837432, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:32.328317 21389 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:50:32.329926 21389 webserver.cc:460] Webserver started at http://127.18.134.62:36769/ using document root <none> and password file <none>
I20260619 02:50:32.330665 21389 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:32.330797 21389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:32.331103 21389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:32.334172 21389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "f0875d1d33044ae39ede5153837004fb"
format_stamp: "Formatted at 2026-06-19 02:50:32 on dist-test-slave-9mr0"
server_key: "bf613ab2f4d30d33fb8e85572a6e0d2f"
server_key_iv: "23bd6b3ed1e8d1be318243d0390784a8"
server_key_version: "encryptionkey@0"
I20260619 02:50:32.334959 21389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "f0875d1d33044ae39ede5153837004fb"
format_stamp: "Formatted at 2026-06-19 02:50:32 on dist-test-slave-9mr0"
server_key: "bf613ab2f4d30d33fb8e85572a6e0d2f"
server_key_iv: "23bd6b3ed1e8d1be318243d0390784a8"
server_key_version: "encryptionkey@0"
I20260619 02:50:32.340270 21389 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260619 02:50:32.343767 21404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:32.345218 21389 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 02:50:32.345391 21389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "f0875d1d33044ae39ede5153837004fb"
format_stamp: "Formatted at 2026-06-19 02:50:32 on dist-test-slave-9mr0"
server_key: "bf613ab2f4d30d33fb8e85572a6e0d2f"
server_key_iv: "23bd6b3ed1e8d1be318243d0390784a8"
server_key_version: "encryptionkey@0"
I20260619 02:50:32.345570 21389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:32.361281 21389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:32.365402 21389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:32.365664 21389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:32.382652 21389 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:45425
I20260619 02:50:32.382690 21455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:45425 every 8 connection(s)
I20260619 02:50:32.384004 21389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:50:32.387859 21456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:32.390597 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 21389
I20260619 02:50:32.390837 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:50:32.391389 18968 external_mini_cluster.cc:1428] Setting key 954b1098def92719d1a4af7d00442705
I20260619 02:50:32.399686 21456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0875d1d33044ae39ede5153837004fb: Bootstrap starting.
I20260619 02:50:32.403542 21456 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0875d1d33044ae39ede5153837004fb: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:32.404817 21456 log.cc:824] T 00000000000000000000000000000000 P f0875d1d33044ae39ede5153837004fb: Log is configured to *not* fsync() on all Append() calls
Jun 19 02:50:32 dist-test-slave-9mr0 krb5kdc[21376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:32.408005 21456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0875d1d33044ae39ede5153837004fb: No bootstrap required, opened a new log
I20260619 02:50:32.413996 21459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:32.393358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60474 (local address 127.18.134.62:45425)
0619 02:50:32.394113 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:32.394125 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:32.394175 (+ 50us) server_negotiation.cc:407] Connection header received
0619 02:50:32.395851 (+ 1676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:32.395874 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:32.396291 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:32.396623 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:32.398261 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:32.399628 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:32.401513 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:32.402200 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:32.402338 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:32.407146 (+ 4808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:32.407154 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:32.407171 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:32.407207 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:32.410057 (+ 2850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:32.410856 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:32.410863 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:32.410869 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:32.411046 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:32.411409 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:32.411412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:32.411414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:32.411920 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:32.412140 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:32.412581 (+ 441us) server_negotiation.cc:299] Negotiation successful
0619 02:50:32.412927 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":340,"threads_started":1}
I20260619 02:50:32.416292 21456 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0875d1d33044ae39ede5153837004fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0875d1d33044ae39ede5153837004fb" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 45425 } }
I20260619 02:50:32.416636 21456 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0875d1d33044ae39ede5153837004fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:32.416734 21456 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0875d1d33044ae39ede5153837004fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0875d1d33044ae39ede5153837004fb, State: Initialized, Role: FOLLOWER
I20260619 02:50:32.417245 21456 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0875d1d33044ae39ede5153837004fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0875d1d33044ae39ede5153837004fb" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 45425 } }
I20260619 02:50:32.417452 21456 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0875d1d33044ae39ede5153837004fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:50:32.417573 21456 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0875d1d33044ae39ede5153837004fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:50:32.417712 21456 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0875d1d33044ae39ede5153837004fb [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:32.419834 21456 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0875d1d33044ae39ede5153837004fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0875d1d33044ae39ede5153837004fb" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 45425 } }
I20260619 02:50:32.420337 21456 leader_election.cc:302] T 00000000000000000000000000000000 P f0875d1d33044ae39ede5153837004fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0875d1d33044ae39ede5153837004fb; no voters:
I20260619 02:50:32.420866 21456 leader_election.cc:288] T 00000000000000000000000000000000 P f0875d1d33044ae39ede5153837004fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:50:32.420974 21461 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0875d1d33044ae39ede5153837004fb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:50:32.421561 21461 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0875d1d33044ae39ede5153837004fb [term 1 LEADER]: Becoming Leader. State: Replica: f0875d1d33044ae39ede5153837004fb, State: Running, Role: LEADER
I20260619 02:50:32.422045 21461 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0875d1d33044ae39ede5153837004fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0875d1d33044ae39ede5153837004fb" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 45425 } }
I20260619 02:50:32.422750 21456 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0875d1d33044ae39ede5153837004fb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:50:32.424407 21462 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0875d1d33044ae39ede5153837004fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0875d1d33044ae39ede5153837004fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0875d1d33044ae39ede5153837004fb" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 45425 } } }
I20260619 02:50:32.424757 21462 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0875d1d33044ae39ede5153837004fb [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:32.425307 21463 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0875d1d33044ae39ede5153837004fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0875d1d33044ae39ede5153837004fb. Latest consensus state: current_term: 1 leader_uuid: "f0875d1d33044ae39ede5153837004fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0875d1d33044ae39ede5153837004fb" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 45425 } } }
I20260619 02:50:32.425648 21463 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0875d1d33044ae39ede5153837004fb [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:32.427388 21466 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:50:32.432705 21466 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:50:32.441001 21466 catalog_manager.cc:1347] Generated new cluster ID: 5630c2895dba44cd89c3751ed7a21505
I20260619 02:50:32.441128 21466 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:50:32.455222 21466 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:50:32.457074 21466 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:50:32.473263 21466 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0875d1d33044ae39ede5153837004fb: Generated new TSK 0
I20260619 02:50:32.474128 21466 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.134.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
I20260619 02:50:32.521446 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:0
--local_ip_for_outbound_sockets=127.18.134.1
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:45425
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:46433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:32.836947 21482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:32.837211 21482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:32.837342 21482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:32.844523 21482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:32.844626 21482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:32.844771 21482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:50:32.854364 21482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:46433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:45425
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:32.855647 21482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:32.856901 21482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:32.869442 21487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:32.869506 21488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:32.869478 21490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:32.870306 21482 server_base.cc:1034] running on GCE node
I20260619 02:50:32.870916 21482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:32.871805 21482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:32.873008 21482 hybrid_clock.cc:648] HybridClock initialized: now 1781837432872966 us; error 35 us; skew 500 ppm
I20260619 02:50:32.873329 21482 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:32 dist-test-slave-9mr0 krb5kdc[21376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837432, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:32.876250 21482 init.cc:377] Logged in from keytab as kudu/127.18.134.1@KRBTEST.COM (short username kudu)
I20260619 02:50:32.877676 21482 webserver.cc:460] Webserver started at http://127.18.134.1:37667/ using document root <none> and password file <none>
I20260619 02:50:32.878325 21482 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:32.878435 21482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:32.878754 21482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:32.881520 21482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/data/instance:
uuid: "1cc4210422a54f9ca5ece3816b951b60"
format_stamp: "Formatted at 2026-06-19 02:50:32 on dist-test-slave-9mr0"
server_key: "c8cf2cb3bb8cf3d23bc79e9e185e5749"
server_key_iv: "e8882299fd0a3a9be44eaacdea3974a5"
server_key_version: "encryptionkey@0"
I20260619 02:50:32.882267 21482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance:
uuid: "1cc4210422a54f9ca5ece3816b951b60"
format_stamp: "Formatted at 2026-06-19 02:50:32 on dist-test-slave-9mr0"
server_key: "c8cf2cb3bb8cf3d23bc79e9e185e5749"
server_key_iv: "e8882299fd0a3a9be44eaacdea3974a5"
server_key_version: "encryptionkey@0"
I20260619 02:50:32.887229 21482 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260619 02:50:32.890604 21497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:32.891964 21482 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:50:32.892112 21482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "1cc4210422a54f9ca5ece3816b951b60"
format_stamp: "Formatted at 2026-06-19 02:50:32 on dist-test-slave-9mr0"
server_key: "c8cf2cb3bb8cf3d23bc79e9e185e5749"
server_key_iv: "e8882299fd0a3a9be44eaacdea3974a5"
server_key_version: "encryptionkey@0"
I20260619 02:50:32.892300 21482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:32.921327 21482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:32.925632 21482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:32.925904 21482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:32.926898 21482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:50:32.928184 21482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:32.928261 21482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:32.928367 21482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:32.928419 21482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:32.957254 21482 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:41431
I20260619 02:50:32.957295 21609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:41431 every 8 connection(s)
I20260619 02:50:32.958470 21482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:50:32.960188 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 21482
I20260619 02:50:32.960368 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance
I20260619 02:50:32.960981 18968 external_mini_cluster.cc:1428] Setting key e2e5069991a6d9f811edb4b432747d63
Jun 19 02:50:32 dist-test-slave-9mr0 krb5kdc[21376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837432, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.134.2@KRBTEST.COM; defaulting to no policy
I20260619 02:50:32.979612 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:32.960934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:55257 (local address 127.18.134.62:45425)
0619 02:50:32.963227 (+ 2293us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:32.963232 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:32.963247 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:50:32.965216 (+ 1969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:32.965227 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:32.965289 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:32.965378 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:32.967332 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:32.968060 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:32.969589 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:32.970182 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:32.970305 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:32.973898 (+ 3593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:32.973903 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:32.973906 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:32.973939 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:32.976400 (+ 2461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:32.977320 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:32.977324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:32.977327 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:32.977446 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:32.977923 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:32.977926 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:32.977928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:32.978279 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:32.978390 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:32.979154 (+ 764us) server_negotiation.cc:299] Negotiation successful
0619 02:50:32.979350 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2199,"thread_start_us":2098,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.2@KRBTEST.COM" created.
I20260619 02:50:32.980664 21613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:32.961613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:45425 (local address 127.18.134.1:55257)
0619 02:50:32.962454 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:32.962680 (+ 226us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:32.964904 (+ 2224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:32.965642 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:32.965655 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:32.966346 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:32.967158 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:32.967182 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:32.968209 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:32.968215 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:32.969445 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:32.969457 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:32.970387 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:32.970391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:32.970591 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:32.971287 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:32.971308 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:32.973655 (+ 2347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:32.976552 (+ 2897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:32.976562 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:32.976584 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:32.977152 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:32.977560 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:32.977565 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:32.977569 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:32.977808 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:32.978392 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:32.978399 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:32.978770 (+ 371us) client_negotiation.cc:769] Sending connection context
0619 02:50:32.979014 (+ 244us) client_negotiation.cc:240] Negotiation successful
0619 02:50:32.979422 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":316,"threads_started":1}
I20260619 02:50:32.982152 21610 heartbeater.cc:347] Connected to a master server at 127.18.134.62:45425
I20260619 02:50:32.982435 21610 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:32.983166 21610 heartbeater.cc:511] Master 127.18.134.62:45425 requested a full tablet report, sending...
I20260619 02:50:32.985373 21421 ts_manager.cc:194] Registered new tserver with Master: 1cc4210422a54f9ca5ece3816b951b60 (127.18.134.1:41431)
I20260619 02:50:32.986842 21421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.1@KRBTEST.COM'} at 127.18.134.1:55257
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
I20260619 02:50:33.004125 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:0
--local_ip_for_outbound_sockets=127.18.134.2
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:45425
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:46433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:33.308754 21616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:33.309023 21616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:33.309121 21616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:33.316227 21616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:33.316334 21616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:33.316444 21616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:50:33.326083 21616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:46433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:45425
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:33.327365 21616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:33.328687 21616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:33.340801 21624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:33.340904 21621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:33.340886 21622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:33.341777 21616 server_base.cc:1034] running on GCE node
I20260619 02:50:33.342358 21616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:33.343313 21616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:33.344519 21616 hybrid_clock.cc:648] HybridClock initialized: now 1781837433344471 us; error 40 us; skew 500 ppm
I20260619 02:50:33.344801 21616 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:33 dist-test-slave-9mr0 krb5kdc[21376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837433, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:33.347862 21616 init.cc:377] Logged in from keytab as kudu/127.18.134.2@KRBTEST.COM (short username kudu)
I20260619 02:50:33.349313 21616 webserver.cc:460] Webserver started at http://127.18.134.2:46531/ using document root <none> and password file <none>
I20260619 02:50:33.349970 21616 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:33.350068 21616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:33.350318 21616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:33.353152 21616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/data/instance:
uuid: "9de7461562a84ffdac8be3235e6415b0"
format_stamp: "Formatted at 2026-06-19 02:50:33 on dist-test-slave-9mr0"
server_key: "a0b00a8f5303cc473230034ea4c1781a"
server_key_iv: "1169cc53622834fe211d534933fccd73"
server_key_version: "encryptionkey@0"
I20260619 02:50:33.353868 21616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance:
uuid: "9de7461562a84ffdac8be3235e6415b0"
format_stamp: "Formatted at 2026-06-19 02:50:33 on dist-test-slave-9mr0"
server_key: "a0b00a8f5303cc473230034ea4c1781a"
server_key_iv: "1169cc53622834fe211d534933fccd73"
server_key_version: "encryptionkey@0"
I20260619 02:50:33.358948 21616 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 02:50:33.362437 21631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:33.363901 21616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:50:33.364056 21616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "9de7461562a84ffdac8be3235e6415b0"
format_stamp: "Formatted at 2026-06-19 02:50:33 on dist-test-slave-9mr0"
server_key: "a0b00a8f5303cc473230034ea4c1781a"
server_key_iv: "1169cc53622834fe211d534933fccd73"
server_key_version: "encryptionkey@0"
I20260619 02:50:33.364285 21616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:33.382601 21616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:33.387748 21616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:33.388031 21616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:33.389112 21616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:50:33.390743 21616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:33.390832 21616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:33.390934 21616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:33.390995 21616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:33.420485 21616 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:46437
I20260619 02:50:33.420527 21743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:46437 every 8 connection(s)
I20260619 02:50:33.421701 21616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:50:33.430100 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 21616
I20260619 02:50:33.430253 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance
I20260619 02:50:33.430651 18968 external_mini_cluster.cc:1428] Setting key 8a9a20a57929e66d181a29648eeb5230
Jun 19 02:50:33 dist-test-slave-9mr0 krb5kdc[21376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837433, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:33.440840 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:33.424091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:41739 (local address 127.18.134.62:45425)
0619 02:50:33.424251 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:33.424255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:33.425405 (+ 1150us) server_negotiation.cc:407] Connection header received
0619 02:50:33.427143 (+ 1738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:33.427148 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:33.427226 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:33.427328 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:33.428805 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:33.429547 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:33.431146 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:33.431639 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:33.431751 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:33.435329 (+ 3578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:33.435335 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:33.435339 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:33.435377 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:33.437986 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:33.438691 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:33.438697 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:33.438700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:33.438847 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:33.439206 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:33.439210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:33.439213 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:33.439509 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:33.439640 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:33.440293 (+ 653us) server_negotiation.cc:299] Negotiation successful
0619 02:50:33.440495 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 02:50:33.441681 21746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:33.424599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:45425 (local address 127.18.134.2:41739)
0619 02:50:33.425257 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:33.425303 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:33.426902 (+ 1599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:33.427544 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:33.427553 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:33.428077 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:33.428652 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:33.428672 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:33.429663 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:33.429666 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:33.431013 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:33.431024 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:33.431863 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:33.431870 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:33.432079 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:33.432701 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:33.432728 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:33.435095 (+ 2367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:33.438123 (+ 3028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:33.438131 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:33.438146 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:33.438553 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:33.438934 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:33.438937 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:33.438939 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:33.439109 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:33.439610 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:33.439616 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:33.439952 (+ 336us) client_negotiation.cc:769] Sending connection context
0619 02:50:33.440180 (+ 228us) client_negotiation.cc:240] Negotiation successful
0619 02:50:33.440552 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":248,"threads_started":1}
I20260619 02:50:33.443028 21744 heartbeater.cc:347] Connected to a master server at 127.18.134.62:45425
I20260619 02:50:33.443302 21744 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:33.443972 21744 heartbeater.cc:511] Master 127.18.134.62:45425 requested a full tablet report, sending...
I20260619 02:50:33.445298 21421 ts_manager.cc:194] Registered new tserver with Master: 9de7461562a84ffdac8be3235e6415b0 (127.18.134.2:46437)
I20260619 02:50:33.446053 21421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.2@KRBTEST.COM'} at 127.18.134.2:41739
WARNING: no policy specified for kudu/127.18.134.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
I20260619 02:50:33.470139 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:0
--local_ip_for_outbound_sockets=127.18.134.3
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:45425
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:46433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:33.782599 21749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:33.782929 21749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:33.783063 21749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:33.791738 21749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:33.791864 21749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:33.792023 21749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:50:33.801841 21749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:46433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:45425
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:33.803190 21749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:33.804512 21749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:33.816516 21754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:33.816540 21755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:33.816581 21757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:33.817703 21749 server_base.cc:1034] running on GCE node
I20260619 02:50:33.818331 21749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:33.819218 21749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:33.820421 21749 hybrid_clock.cc:648] HybridClock initialized: now 1781837433820361 us; error 47 us; skew 500 ppm
I20260619 02:50:33.820724 21749 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:33 dist-test-slave-9mr0 krb5kdc[21376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837433, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:33.823751 21749 init.cc:377] Logged in from keytab as kudu/127.18.134.3@KRBTEST.COM (short username kudu)
I20260619 02:50:33.825111 21749 webserver.cc:460] Webserver started at http://127.18.134.3:33861/ using document root <none> and password file <none>
I20260619 02:50:33.825764 21749 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:33.825872 21749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:33.826153 21749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:33.829018 21749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/data/instance:
uuid: "fe37e68e304142a099b3946bc42a8cbb"
format_stamp: "Formatted at 2026-06-19 02:50:33 on dist-test-slave-9mr0"
server_key: "6c7671249993fd3a617066cf5a10281e"
server_key_iv: "7c8c6a2641b2cf9b3c24ee3fe4d88c09"
server_key_version: "encryptionkey@0"
I20260619 02:50:33.829747 21749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance:
uuid: "fe37e68e304142a099b3946bc42a8cbb"
format_stamp: "Formatted at 2026-06-19 02:50:33 on dist-test-slave-9mr0"
server_key: "6c7671249993fd3a617066cf5a10281e"
server_key_iv: "7c8c6a2641b2cf9b3c24ee3fe4d88c09"
server_key_version: "encryptionkey@0"
I20260619 02:50:33.834654 21749 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260619 02:50:33.837960 21764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:33.839329 21749 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260619 02:50:33.839476 21749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "fe37e68e304142a099b3946bc42a8cbb"
format_stamp: "Formatted at 2026-06-19 02:50:33 on dist-test-slave-9mr0"
server_key: "6c7671249993fd3a617066cf5a10281e"
server_key_iv: "7c8c6a2641b2cf9b3c24ee3fe4d88c09"
server_key_version: "encryptionkey@0"
I20260619 02:50:33.839645 21749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:33.864365 21749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:33.868508 21749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:33.868762 21749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:33.869696 21749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:50:33.870975 21749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:33.871048 21749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:33.871167 21749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:33.871218 21749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:33.899892 21749 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:37595
I20260619 02:50:33.899928 21876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:37595 every 8 connection(s)
I20260619 02:50:33.901084 21749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:50:33.905905 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 21749
I20260619 02:50:33.906070 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance
I20260619 02:50:33.906467 18968 external_mini_cluster.cc:1428] Setting key 465c5b0eb3b9d7104b5a4ce5703a0234
Jun 19 02:50:33 dist-test-slave-9mr0 krb5kdc[21376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837433, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:33.919839 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:33.903409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:45315 (local address 127.18.134.62:45425)
0619 02:50:33.903600 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:33.903607 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:33.904715 (+ 1108us) server_negotiation.cc:407] Connection header received
0619 02:50:33.906501 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:33.906506 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:33.906589 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:33.906717 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:33.908406 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:33.909441 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:33.910945 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:33.911337 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:33.911444 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:33.914603 (+ 3159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:33.914608 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:33.914611 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:33.914676 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:33.917081 (+ 2405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:33.917727 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:33.917733 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:33.917737 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:33.917848 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:33.918224 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:33.918229 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:33.918233 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:33.918498 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:33.918620 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:33.919279 (+ 659us) server_negotiation.cc:299] Negotiation successful
0619 02:50:33.919491 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 02:50:33.920907 21879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:33.903907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:45425 (local address 127.18.134.3:45315)
0619 02:50:33.904574 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:33.904610 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:33.906251 (+ 1641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:33.906929 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:33.906941 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:33.907594 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:33.908257 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:33.908277 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:33.909573 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:33.909578 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:33.910812 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:33.910823 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:33.911538 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:33.911545 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:33.911777 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:33.912382 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:33.912403 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:33.914400 (+ 1997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:33.917202 (+ 2802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:33.917220 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:33.917236 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:33.917612 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:33.917943 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:33.917948 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:33.917952 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:33.918126 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:33.918620 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:33.918650 (+ 30us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:33.918955 (+ 305us) client_negotiation.cc:769] Sending connection context
0619 02:50:33.919189 (+ 234us) client_negotiation.cc:240] Negotiation successful
0619 02:50:33.919682 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":257,"threads_started":1}
I20260619 02:50:33.922176 21877 heartbeater.cc:347] Connected to a master server at 127.18.134.62:45425
I20260619 02:50:33.922453 21877 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:33.923175 21877 heartbeater.cc:511] Master 127.18.134.62:45425 requested a full tablet report, sending...
I20260619 02:50:33.924495 21421 ts_manager.cc:194] Registered new tserver with Master: fe37e68e304142a099b3946bc42a8cbb (127.18.134.3:37595)
I20260619 02:50:33.925242 21421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.3@KRBTEST.COM'} at 127.18.134.3:45315
I20260619 02:50:33.933773 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 02:50:33.961366 21612 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:50:33.959058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60476 (local address 127.18.134.62:45425)
0619 02:50:33.959234 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:33.959241 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:33.959642 (+ 401us) server_negotiation.cc:407] Connection header received
0619 02:50:33.959806 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:33.959822 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:33.959897 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:33.959981 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:33.961150 (+ 1169us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60476: BlockingRecv error: recv got EOF from 127.0.0.1:60476 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 02:50:33.964236 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 21482
W20260619 02:50:33.966909 18968 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 21482: Not authorized: Client connection negotiation failed: client connection to 127.18.134.1:41431: server requires authentication, but client does not have Kerberos credentials available
I20260619 02:50:33.967092 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 21482
I20260619 02:50:33.979120 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 21616
W20260619 02:50:33.981776 18968 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 21616: Not authorized: Client connection negotiation failed: client connection to 127.18.134.2:46437: server requires authentication, but client does not have Kerberos credentials available
I20260619 02:50:33.981940 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 21616
I20260619 02:50:33.993462 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 21749
W20260619 02:50:33.996275 18968 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 21749: Not authorized: Client connection negotiation failed: client connection to 127.18.134.3:37595: server requires authentication, but client does not have Kerberos credentials available
I20260619 02:50:33.996424 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 21749
I20260619 02:50:34.007725 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 21389
I20260619 02:50:34.008767 21451 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:34.107033 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 21389
2026-06-19T02:50:34Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3302 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:50:34 dist-test-slave-9mr0 krb5kdc[21894](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:50:34 dist-test-slave-9mr0 krb5kdc[21894](info): set up 2 sockets
Jun 19 02:50:34 dist-test-slave-9mr0 krb5kdc[21894](info): commencing operation
krb5kdc: starting...
W20260619 02:50:35.102607 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.959s user 0.004s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:50:35 dist-test-slave-9mr0 krb5kdc[21894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:50:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:50:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:50:35.277803 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:42821
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:34811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:42821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:35.578529 21907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:35.578819 21907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:35.578944 21907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:35.586004 21907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:50:35.586110 21907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:35.586177 21907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:35.586254 21907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:50:35.586302 21907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:50:35.595865 21907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:34811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:42821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:42821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:35.597075 21907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:35.598328 21907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:35.609419 21912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:35.609452 21915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:35.609520 21913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:35.610213 21907 server_base.cc:1034] running on GCE node
I20260619 02:50:35.610993 21907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:35.612363 21907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:35.613562 21907 hybrid_clock.cc:648] HybridClock initialized: now 1781837435613509 us; error 41 us; skew 500 ppm
I20260619 02:50:35.613880 21907 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:35 dist-test-slave-9mr0 krb5kdc[21894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837435, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:35.616750 21907 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:50:35.618018 21907 webserver.cc:460] Webserver started at http://127.18.134.62:40571/ using document root <none> and password file <none>
I20260619 02:50:35.618674 21907 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:35.618777 21907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:35.619021 21907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:35.621781 21907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "d19f92e7c8ce40a39745664cd7c713b4"
format_stamp: "Formatted at 2026-06-19 02:50:35 on dist-test-slave-9mr0"
server_key: "21365897433a6751c75abafaa3858db3"
server_key_iv: "7837186fd642a5c37aa904c8bd0a4a9e"
server_key_version: "encryptionkey@0"
I20260619 02:50:35.622529 21907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "d19f92e7c8ce40a39745664cd7c713b4"
format_stamp: "Formatted at 2026-06-19 02:50:35 on dist-test-slave-9mr0"
server_key: "21365897433a6751c75abafaa3858db3"
server_key_iv: "7837186fd642a5c37aa904c8bd0a4a9e"
server_key_version: "encryptionkey@0"
I20260619 02:50:35.627368 21907 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 02:50:35.630573 21922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:35.631923 21907 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260619 02:50:35.632118 21907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "d19f92e7c8ce40a39745664cd7c713b4"
format_stamp: "Formatted at 2026-06-19 02:50:35 on dist-test-slave-9mr0"
server_key: "21365897433a6751c75abafaa3858db3"
server_key_iv: "7837186fd642a5c37aa904c8bd0a4a9e"
server_key_version: "encryptionkey@0"
I20260619 02:50:35.632302 21907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:35.657567 21907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:35.661398 21907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:35.661617 21907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:35.678934 21907 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:42821
I20260619 02:50:35.678985 21973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:42821 every 8 connection(s)
I20260619 02:50:35.680259 21907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:50:35.683429 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 21907
I20260619 02:50:35.683631 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:50:35.684042 21974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:35.684281 18968 external_mini_cluster.cc:1428] Setting key 0b1c72bd69104d7bed7090d089afa799
I20260619 02:50:35.698172 21974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d19f92e7c8ce40a39745664cd7c713b4: Bootstrap starting.
Jun 19 02:50:35 dist-test-slave-9mr0 krb5kdc[21894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:35.701895 21974 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d19f92e7c8ce40a39745664cd7c713b4: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:35.703265 21974 log.cc:824] T 00000000000000000000000000000000 P d19f92e7c8ce40a39745664cd7c713b4: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:35.706044 21974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d19f92e7c8ce40a39745664cd7c713b4: No bootstrap required, opened a new log
I20260619 02:50:35.707396 21977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:35.685957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57408 (local address 127.18.134.62:42821)
0619 02:50:35.686835 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:35.686853 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:35.686907 (+ 54us) server_negotiation.cc:407] Connection header received
0619 02:50:35.688762 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:35.688793 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:35.689398 (+ 605us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:35.689912 (+ 514us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:35.691313 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:35.693208 (+ 1895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:35.694712 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:35.695807 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:35.696001 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:35.699761 (+ 3760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:35.699771 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:35.699805 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:35.699856 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:35.703444 (+ 3588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:35.704083 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:35.704091 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:35.704100 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:35.704311 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:35.704632 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:35.704636 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:35.704638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:35.705192 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:35.705454 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:35.705904 (+ 450us) server_negotiation.cc:299] Negotiation successful
0619 02:50:35.706360 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":333,"threads_started":1}
I20260619 02:50:35.711088 18968 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:50:35.713663 21974 raft_consensus.cc:348] T 00000000000000000000000000000000 P d19f92e7c8ce40a39745664cd7c713b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d19f92e7c8ce40a39745664cd7c713b4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 42821 } }
I20260619 02:50:35.714018 21974 raft_consensus.cc:374] T 00000000000000000000000000000000 P d19f92e7c8ce40a39745664cd7c713b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:35.714104 21974 raft_consensus.cc:729] T 00000000000000000000000000000000 P d19f92e7c8ce40a39745664cd7c713b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d19f92e7c8ce40a39745664cd7c713b4, State: Initialized, Role: FOLLOWER
I20260619 02:50:35.714656 21974 consensus_queue.cc:260] T 00000000000000000000000000000000 P d19f92e7c8ce40a39745664cd7c713b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d19f92e7c8ce40a39745664cd7c713b4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 42821 } }
I20260619 02:50:35.714882 21974 raft_consensus.cc:388] T 00000000000000000000000000000000 P d19f92e7c8ce40a39745664cd7c713b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:50:35.714998 21974 raft_consensus.cc:482] T 00000000000000000000000000000000 P d19f92e7c8ce40a39745664cd7c713b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:50:35.715154 21974 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d19f92e7c8ce40a39745664cd7c713b4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:35.717178 21974 raft_consensus.cc:504] T 00000000000000000000000000000000 P d19f92e7c8ce40a39745664cd7c713b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d19f92e7c8ce40a39745664cd7c713b4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 42821 } }
I20260619 02:50:35.717720 21974 leader_election.cc:302] T 00000000000000000000000000000000 P d19f92e7c8ce40a39745664cd7c713b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d19f92e7c8ce40a39745664cd7c713b4; no voters:
I20260619 02:50:35.718297 21974 leader_election.cc:288] T 00000000000000000000000000000000 P d19f92e7c8ce40a39745664cd7c713b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:50:35.718415 21980 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d19f92e7c8ce40a39745664cd7c713b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:50:35.720336 21974 sys_catalog.cc:564] T 00000000000000000000000000000000 P d19f92e7c8ce40a39745664cd7c713b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:50:35.722903 21980 raft_consensus.cc:686] T 00000000000000000000000000000000 P d19f92e7c8ce40a39745664cd7c713b4 [term 1 LEADER]: Becoming Leader. State: Replica: d19f92e7c8ce40a39745664cd7c713b4, State: Running, Role: LEADER
I20260619 02:50:35.723518 21980 consensus_queue.cc:237] T 00000000000000000000000000000000 P d19f92e7c8ce40a39745664cd7c713b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d19f92e7c8ce40a39745664cd7c713b4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 42821 } }
I20260619 02:50:35.726658 21982 sys_catalog.cc:455] T 00000000000000000000000000000000 P d19f92e7c8ce40a39745664cd7c713b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d19f92e7c8ce40a39745664cd7c713b4. Latest consensus state: current_term: 1 leader_uuid: "d19f92e7c8ce40a39745664cd7c713b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d19f92e7c8ce40a39745664cd7c713b4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 42821 } } }
I20260619 02:50:35.727015 21982 sys_catalog.cc:458] T 00000000000000000000000000000000 P d19f92e7c8ce40a39745664cd7c713b4 [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:35.727515 21981 sys_catalog.cc:455] T 00000000000000000000000000000000 P d19f92e7c8ce40a39745664cd7c713b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d19f92e7c8ce40a39745664cd7c713b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d19f92e7c8ce40a39745664cd7c713b4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 42821 } } }
I20260619 02:50:35.727783 21981 sys_catalog.cc:458] T 00000000000000000000000000000000 P d19f92e7c8ce40a39745664cd7c713b4 [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:35.731595 21992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:50:35.739250 21992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:50:35.749046 21992 catalog_manager.cc:1347] Generated new cluster ID: 857404fc9fbf48ccb426d8a5bbe385fa
I20260619 02:50:35.749167 21992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:50:35.756165 21977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:35.749752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57424 (local address 127.18.134.62:42821)
0619 02:50:35.749910 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:35.749915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:35.749934 (+ 19us) server_negotiation.cc:407] Connection header received
0619 02:50:35.749996 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:35.750001 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:35.750064 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:35.750143 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:35.751251 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:35.751996 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:35.753252 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:35.753664 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:35.753772 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:35.754807 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:35.754812 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:35.754815 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:50:35.755428 (+ 613us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:35.755469 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:50:35.755499 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:35.755591 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:35.755735 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 02:50:35.755827 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 02:50:35.779364 21992 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:50:35.781314 21992 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:50:35.789882 21992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d19f92e7c8ce40a39745664cd7c713b4: Generated new TSK 0
I20260619 02:50:35.790760 21992 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:50:35.803175 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 21907
I20260619 02:50:35.804445 21969 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:35.892266 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 21907
2026-06-19T02:50:35Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1779 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:50:35 dist-test-slave-9mr0 krb5kdc[22009](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:50:35 dist-test-slave-9mr0 krb5kdc[22009](info): set up 2 sockets
Jun 19 02:50:35 dist-test-slave-9mr0 krb5kdc[22009](info): commencing operation
krb5kdc: starting...
W20260619 02:50:36.882725 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.959s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:50:36 dist-test-slave-9mr0 krb5kdc[22009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:50:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:50:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:50:37.060674 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:35417
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:35043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:35417
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:37.364595 22022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:37.364887 22022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:37.365003 22022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:37.372262 22022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:50:37.372402 22022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:37.372481 22022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:37.372567 22022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:50:37.372627 22022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:50:37.382416 22022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:35043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:35417
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:35417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:37.383769 22022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:37.385005 22022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:37.395857 22028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:37.396044 22030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:37.396219 22027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:37.396716 22022 server_base.cc:1034] running on GCE node
I20260619 02:50:37.397430 22022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:37.398834 22022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:37.400048 22022 hybrid_clock.cc:648] HybridClock initialized: now 1781837437399990 us; error 44 us; skew 500 ppm
I20260619 02:50:37.400377 22022 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:37 dist-test-slave-9mr0 krb5kdc[22009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837437, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:37.407196 22022 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:50:37.408636 22022 webserver.cc:460] Webserver started at http://127.18.134.62:36295/ using document root <none> and password file <none>
I20260619 02:50:37.409310 22022 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:37.409426 22022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:37.409705 22022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:37.412647 22022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "194e05cd16b34715ad5ed15d9a433779"
format_stamp: "Formatted at 2026-06-19 02:50:37 on dist-test-slave-9mr0"
server_key: "e998f7aab5da62f6635673993b5ed52d"
server_key_iv: "e70e09459f835e9f30ef7806c5942987"
server_key_version: "encryptionkey@0"
I20260619 02:50:37.413364 22022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "194e05cd16b34715ad5ed15d9a433779"
format_stamp: "Formatted at 2026-06-19 02:50:37 on dist-test-slave-9mr0"
server_key: "e998f7aab5da62f6635673993b5ed52d"
server_key_iv: "e70e09459f835e9f30ef7806c5942987"
server_key_version: "encryptionkey@0"
I20260619 02:50:37.418390 22022 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260619 02:50:37.421871 22037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:37.423262 22022 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:50:37.423429 22022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "194e05cd16b34715ad5ed15d9a433779"
format_stamp: "Formatted at 2026-06-19 02:50:37 on dist-test-slave-9mr0"
server_key: "e998f7aab5da62f6635673993b5ed52d"
server_key_iv: "e70e09459f835e9f30ef7806c5942987"
server_key_version: "encryptionkey@0"
I20260619 02:50:37.423606 22022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:37.434089 22022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:37.438072 22022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:37.438354 22022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:37.454546 22022 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:35417
I20260619 02:50:37.454563 22088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:35417 every 8 connection(s)
I20260619 02:50:37.455901 22022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:50:37.456928 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 22022
I20260619 02:50:37.457134 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:50:37.457909 18968 external_mini_cluster.cc:1428] Setting key c3b2dd809ff048dc497c59b31174ff07
I20260619 02:50:37.460489 22089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 19 02:50:37 dist-test-slave-9mr0 krb5kdc[22009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:37.472635 22089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 194e05cd16b34715ad5ed15d9a433779: Bootstrap starting.
I20260619 02:50:37.475761 22089 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 194e05cd16b34715ad5ed15d9a433779: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:37.476868 22089 log.cc:824] T 00000000000000000000000000000000 P 194e05cd16b34715ad5ed15d9a433779: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:37.479954 22089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 194e05cd16b34715ad5ed15d9a433779: No bootstrap required, opened a new log
I20260619 02:50:37.480334 22092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:37.459842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38332 (local address 127.18.134.62:35417)
0619 02:50:37.460580 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:37.460594 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:37.460641 (+ 47us) server_negotiation.cc:407] Connection header received
0619 02:50:37.462779 (+ 2138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:37.462805 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:37.463279 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:37.463722 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:37.464907 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:37.466955 (+ 2048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:37.468224 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:37.469226 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:37.469407 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:37.473380 (+ 3973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:37.473388 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:37.473408 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:37.473447 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:37.476302 (+ 2855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:37.477077 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:37.477084 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:37.477089 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:37.477244 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:37.477658 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:37.477663 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:37.477670 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:37.478292 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:37.478504 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:37.478959 (+ 455us) server_negotiation.cc:299] Negotiation successful
0619 02:50:37.479330 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":323,"threads_started":1}
I20260619 02:50:37.487679 22089 raft_consensus.cc:348] T 00000000000000000000000000000000 P 194e05cd16b34715ad5ed15d9a433779 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "194e05cd16b34715ad5ed15d9a433779" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35417 } }
I20260619 02:50:37.488003 22089 raft_consensus.cc:374] T 00000000000000000000000000000000 P 194e05cd16b34715ad5ed15d9a433779 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:37.488091 22089 raft_consensus.cc:729] T 00000000000000000000000000000000 P 194e05cd16b34715ad5ed15d9a433779 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 194e05cd16b34715ad5ed15d9a433779, State: Initialized, Role: FOLLOWER
I20260619 02:50:37.488600 22089 consensus_queue.cc:260] T 00000000000000000000000000000000 P 194e05cd16b34715ad5ed15d9a433779 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "194e05cd16b34715ad5ed15d9a433779" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35417 } }
I20260619 02:50:37.488797 22089 raft_consensus.cc:388] T 00000000000000000000000000000000 P 194e05cd16b34715ad5ed15d9a433779 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:50:37.488910 22089 raft_consensus.cc:482] T 00000000000000000000000000000000 P 194e05cd16b34715ad5ed15d9a433779 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:50:37.489045 22089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 194e05cd16b34715ad5ed15d9a433779 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:37.491245 22089 raft_consensus.cc:504] T 00000000000000000000000000000000 P 194e05cd16b34715ad5ed15d9a433779 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "194e05cd16b34715ad5ed15d9a433779" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35417 } }
I20260619 02:50:37.491700 22089 leader_election.cc:302] T 00000000000000000000000000000000 P 194e05cd16b34715ad5ed15d9a433779 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 194e05cd16b34715ad5ed15d9a433779; no voters:
I20260619 02:50:37.492259 22089 leader_election.cc:288] T 00000000000000000000000000000000 P 194e05cd16b34715ad5ed15d9a433779 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:50:37.492383 22094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 194e05cd16b34715ad5ed15d9a433779 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:50:37.492945 22094 raft_consensus.cc:686] T 00000000000000000000000000000000 P 194e05cd16b34715ad5ed15d9a433779 [term 1 LEADER]: Becoming Leader. State: Replica: 194e05cd16b34715ad5ed15d9a433779, State: Running, Role: LEADER
I20260619 02:50:37.493498 22094 consensus_queue.cc:237] T 00000000000000000000000000000000 P 194e05cd16b34715ad5ed15d9a433779 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "194e05cd16b34715ad5ed15d9a433779" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35417 } }
I20260619 02:50:37.494130 22089 sys_catalog.cc:564] T 00000000000000000000000000000000 P 194e05cd16b34715ad5ed15d9a433779 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:50:37.497718 22095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 194e05cd16b34715ad5ed15d9a433779 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "194e05cd16b34715ad5ed15d9a433779" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "194e05cd16b34715ad5ed15d9a433779" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35417 } } }
I20260619 02:50:37.500427 22095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 194e05cd16b34715ad5ed15d9a433779 [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:37.497691 22096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 194e05cd16b34715ad5ed15d9a433779 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 194e05cd16b34715ad5ed15d9a433779. Latest consensus state: current_term: 1 leader_uuid: "194e05cd16b34715ad5ed15d9a433779" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "194e05cd16b34715ad5ed15d9a433779" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35417 } } }
I20260619 02:50:37.500862 22096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 194e05cd16b34715ad5ed15d9a433779 [sys.catalog]: This master's current role is: LEADER
W20260619 02:50:37.506917 22108 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 194e05cd16b34715ad5ed15d9a433779: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:50:37.507025 22108 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:50:37.507122 22110 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:50:37.509092 22110 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:50:37.517079 22110 catalog_manager.cc:1347] Generated new cluster ID: 9b759095583542ec8e923f0a5011e58d
I20260619 02:50:37.517190 22110 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:50:37.547984 22110 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:50:37.549695 22110 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:50:37.555915 22110 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 194e05cd16b34715ad5ed15d9a433779: Generated new TSK 0
I20260619 02:50:37.556730 22110 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.134.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
I20260619 02:50:37.603309 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:0
--local_ip_for_outbound_sockets=127.18.134.1
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:35417
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:35043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:37.911711 22115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:37.911990 22115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:37.912110 22115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:37.919278 22115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:37.919400 22115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:37.919552 22115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:50:37.929236 22115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:35043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:35417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:37.930599 22115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:37.932047 22115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:37.944926 22120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:37.944885 22121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:37.944949 22123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:37.945616 22115 server_base.cc:1034] running on GCE node
I20260619 02:50:37.946205 22115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:37.947158 22115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:37.948372 22115 hybrid_clock.cc:648] HybridClock initialized: now 1781837437948321 us; error 47 us; skew 500 ppm
I20260619 02:50:37.948668 22115 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:37 dist-test-slave-9mr0 krb5kdc[22009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837437, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:37.951653 22115 init.cc:377] Logged in from keytab as kudu/127.18.134.1@KRBTEST.COM (short username kudu)
I20260619 02:50:37.953007 22115 webserver.cc:460] Webserver started at http://127.18.134.1:43163/ using document root <none> and password file <none>
I20260619 02:50:37.953649 22115 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:37.953787 22115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:37.954068 22115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:37.956949 22115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/data/instance:
uuid: "2ad04d20d9a24a618ce8b143aec6904e"
format_stamp: "Formatted at 2026-06-19 02:50:37 on dist-test-slave-9mr0"
server_key: "bc43da19bfbe500abcacaf36f23c6ab9"
server_key_iv: "f6d9ebd2ad80839c489ef3ee76fd5314"
server_key_version: "encryptionkey@0"
I20260619 02:50:37.957696 22115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance:
uuid: "2ad04d20d9a24a618ce8b143aec6904e"
format_stamp: "Formatted at 2026-06-19 02:50:37 on dist-test-slave-9mr0"
server_key: "bc43da19bfbe500abcacaf36f23c6ab9"
server_key_iv: "f6d9ebd2ad80839c489ef3ee76fd5314"
server_key_version: "encryptionkey@0"
I20260619 02:50:37.962590 22115 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260619 02:50:37.965888 22130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:37.967272 22115 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:50:37.967413 22115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "2ad04d20d9a24a618ce8b143aec6904e"
format_stamp: "Formatted at 2026-06-19 02:50:37 on dist-test-slave-9mr0"
server_key: "bc43da19bfbe500abcacaf36f23c6ab9"
server_key_iv: "f6d9ebd2ad80839c489ef3ee76fd5314"
server_key_version: "encryptionkey@0"
I20260619 02:50:37.967576 22115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:37.986158 22115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:37.989876 22115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:37.990124 22115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:37.991032 22115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:50:37.992287 22115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:37.992362 22115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:37.992475 22115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:37.992525 22115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:38.021341 22115 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:37895
I20260619 02:50:38.021377 22242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:37895 every 8 connection(s)
I20260619 02:50:38.022619 22115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:50:38.031231 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 22115
I20260619 02:50:38.031432 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance
I20260619 02:50:38.032178 18968 external_mini_cluster.cc:1428] Setting key 9669f03395947a209686851cd8164093
Jun 19 02:50:38 dist-test-slave-9mr0 krb5kdc[22009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837437, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:38.043973 22245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:38.025007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:49431 (local address 127.18.134.62:35417)
0619 02:50:38.025386 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:38.025391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:38.026402 (+ 1011us) server_negotiation.cc:407] Connection header received
0619 02:50:38.028199 (+ 1797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:38.028204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:38.028295 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:38.028388 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:38.029858 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:38.030535 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:38.032799 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:38.033375 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:38.033503 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:38.037609 (+ 4106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:38.037616 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:38.037619 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:38.037659 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:38.040392 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:38.041307 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:38.041312 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:38.041315 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:38.041450 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:38.041981 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:38.041985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:38.041988 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:38.042355 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:38.042474 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:38.043516 (+ 1042us) server_negotiation.cc:299] Negotiation successful
0619 02:50:38.043723 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":180,"threads_started":1}
I20260619 02:50:38.045246 22246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:38.025585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35417 (local address 127.18.134.1:49431)
0619 02:50:38.026243 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:38.026292 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:38.027928 (+ 1636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:38.028636 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:38.028644 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:38.029135 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:38.029694 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:38.029711 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:38.030705 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:38.030729 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:38.032631 (+ 1902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:38.032645 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:38.033573 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:38.033578 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:38.033778 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:38.034459 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:38.034496 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:38.037292 (+ 2796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:38.040553 (+ 3261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:38.040563 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:38.040583 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:38.041152 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:38.041578 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:38.041582 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:38.041586 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:38.041823 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:38.042474 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:38.042483 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:38.042943 (+ 460us) client_negotiation.cc:769] Sending connection context
0619 02:50:38.043267 (+ 324us) client_negotiation.cc:240] Negotiation successful
0619 02:50:38.043793 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":237,"threads_started":1}
I20260619 02:50:38.046929 22243 heartbeater.cc:347] Connected to a master server at 127.18.134.62:35417
I20260619 02:50:38.047220 22243 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:38.047959 22243 heartbeater.cc:511] Master 127.18.134.62:35417 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.134.2@KRBTEST.COM; defaulting to no policy
I20260619 02:50:38.050482 22054 ts_manager.cc:194] Registered new tserver with Master: 2ad04d20d9a24a618ce8b143aec6904e (127.18.134.1:37895)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.2@KRBTEST.COM" created.
I20260619 02:50:38.052106 22054 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.1@KRBTEST.COM'} at 127.18.134.1:49431
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
I20260619 02:50:38.073429 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:0
--local_ip_for_outbound_sockets=127.18.134.2
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:35417
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:35043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:38.380033 22249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:38.380295 22249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:38.380384 22249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:38.387815 22249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:38.387933 22249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:38.388077 22249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:50:38.397547 22249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:35043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:35417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:38.398859 22249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:38.400102 22249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:38.412580 22255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:38.412701 22254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:38.412765 22257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:38.413313 22249 server_base.cc:1034] running on GCE node
I20260619 02:50:38.413930 22249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:38.414918 22249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:38.416141 22249 hybrid_clock.cc:648] HybridClock initialized: now 1781837438416078 us; error 55 us; skew 500 ppm
I20260619 02:50:38.416483 22249 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:38 dist-test-slave-9mr0 krb5kdc[22009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837438, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:38.419726 22249 init.cc:377] Logged in from keytab as kudu/127.18.134.2@KRBTEST.COM (short username kudu)
I20260619 02:50:38.421145 22249 webserver.cc:460] Webserver started at http://127.18.134.2:36541/ using document root <none> and password file <none>
I20260619 02:50:38.421794 22249 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:38.421907 22249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:38.422174 22249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:38.424939 22249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/data/instance:
uuid: "0650211843dc4f5bb5a326aacd2e23d1"
format_stamp: "Formatted at 2026-06-19 02:50:38 on dist-test-slave-9mr0"
server_key: "9a2e8821dd67c44e784f96bde5b86f28"
server_key_iv: "f8b020cb0807dddabebc637b75e45694"
server_key_version: "encryptionkey@0"
I20260619 02:50:38.425643 22249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance:
uuid: "0650211843dc4f5bb5a326aacd2e23d1"
format_stamp: "Formatted at 2026-06-19 02:50:38 on dist-test-slave-9mr0"
server_key: "9a2e8821dd67c44e784f96bde5b86f28"
server_key_iv: "f8b020cb0807dddabebc637b75e45694"
server_key_version: "encryptionkey@0"
I20260619 02:50:38.430483 22249 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260619 02:50:38.433590 22264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:38.434842 22249 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 02:50:38.434983 22249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "0650211843dc4f5bb5a326aacd2e23d1"
format_stamp: "Formatted at 2026-06-19 02:50:38 on dist-test-slave-9mr0"
server_key: "9a2e8821dd67c44e784f96bde5b86f28"
server_key_iv: "f8b020cb0807dddabebc637b75e45694"
server_key_version: "encryptionkey@0"
I20260619 02:50:38.435151 22249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:38.451167 22249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:38.455001 22249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:38.455284 22249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:38.456355 22249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:50:38.457933 22249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:38.458017 22249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:38.458127 22249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:38.458178 22249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:38.490478 22249 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:42997
I20260619 02:50:38.490499 22376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:42997 every 8 connection(s)
I20260619 02:50:38.491742 22249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:50:38.499918 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 22249
I20260619 02:50:38.500072 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance
I20260619 02:50:38.500490 18968 external_mini_cluster.cc:1428] Setting key b004a20bf74dee645265bc97cf924502
Jun 19 02:50:38 dist-test-slave-9mr0 krb5kdc[22009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837438, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:38.510174 22245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:38.494039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:46229 (local address 127.18.134.62:35417)
0619 02:50:38.494250 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:38.494256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:38.495414 (+ 1158us) server_negotiation.cc:407] Connection header received
0619 02:50:38.497132 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:38.497136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:38.497226 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:38.497325 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:38.498738 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:38.499450 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:38.500989 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:38.501540 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:38.501661 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:38.505010 (+ 3349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:38.505015 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:38.505018 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:38.505048 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:38.507411 (+ 2363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:38.508093 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:38.508099 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:38.508103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:38.508243 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:38.508604 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:38.508608 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:38.508611 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:38.508923 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:38.509050 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:38.509659 (+ 609us) server_negotiation.cc:299] Negotiation successful
0619 02:50:38.509867 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 02:50:38.511000 22379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:38.494576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35417 (local address 127.18.134.2:46229)
0619 02:50:38.495273 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:38.495326 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:38.496848 (+ 1522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:38.497515 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:38.497523 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:38.498038 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:38.498575 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:38.498592 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:38.499591 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:38.499595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:38.500869 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:38.500880 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:38.501722 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:38.501726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:38.501918 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:38.502590 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:38.502615 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:38.504822 (+ 2207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:38.507526 (+ 2704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:38.507535 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:38.507551 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:38.507966 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:38.508339 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:38.508342 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:38.508344 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:38.508513 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:38.509023 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:38.509028 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:38.509327 (+ 299us) client_negotiation.cc:769] Sending connection context
0619 02:50:38.509542 (+ 215us) client_negotiation.cc:240] Negotiation successful
0619 02:50:38.509900 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":246,"threads_started":1}
I20260619 02:50:38.512355 22377 heartbeater.cc:347] Connected to a master server at 127.18.134.62:35417
I20260619 02:50:38.512616 22377 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:38.513329 22377 heartbeater.cc:511] Master 127.18.134.62:35417 requested a full tablet report, sending...
I20260619 02:50:38.514863 22054 ts_manager.cc:194] Registered new tserver with Master: 0650211843dc4f5bb5a326aacd2e23d1 (127.18.134.2:42997)
I20260619 02:50:38.515640 22054 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.2@KRBTEST.COM'} at 127.18.134.2:46229
WARNING: no policy specified for kudu/127.18.134.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
I20260619 02:50:38.541913 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:0
--local_ip_for_outbound_sockets=127.18.134.3
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:35417
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:35043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:38.848315 22382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:38.848584 22382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:38.848681 22382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:38.855770 22382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:38.855878 22382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:38.855988 22382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:50:38.865499 22382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:35043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:35417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:38.866822 22382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:38.868028 22382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:38.880494 22387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:38.880426 22388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:38.880724 22390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:38.881474 22382 server_base.cc:1034] running on GCE node
I20260619 02:50:38.882084 22382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:38.883060 22382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:38.884286 22382 hybrid_clock.cc:648] HybridClock initialized: now 1781837438884235 us; error 36 us; skew 500 ppm
I20260619 02:50:38.884613 22382 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:38 dist-test-slave-9mr0 krb5kdc[22009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837438, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:38.887809 22382 init.cc:377] Logged in from keytab as kudu/127.18.134.3@KRBTEST.COM (short username kudu)
I20260619 02:50:38.889214 22382 webserver.cc:460] Webserver started at http://127.18.134.3:32859/ using document root <none> and password file <none>
I20260619 02:50:38.889925 22382 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:38.890061 22382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:38.890340 22382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:38.893231 22382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/data/instance:
uuid: "c31832b6306840f1866236fa7587a8f3"
format_stamp: "Formatted at 2026-06-19 02:50:38 on dist-test-slave-9mr0"
server_key: "8985f0f273e815d12b125ad5e0a2091d"
server_key_iv: "9554b1849f3cab01b1a64f3ef2679044"
server_key_version: "encryptionkey@0"
I20260619 02:50:38.894040 22382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance:
uuid: "c31832b6306840f1866236fa7587a8f3"
format_stamp: "Formatted at 2026-06-19 02:50:38 on dist-test-slave-9mr0"
server_key: "8985f0f273e815d12b125ad5e0a2091d"
server_key_iv: "9554b1849f3cab01b1a64f3ef2679044"
server_key_version: "encryptionkey@0"
I20260619 02:50:38.899175 22382 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260619 02:50:38.902470 22397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:38.903914 22382 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:50:38.904083 22382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "c31832b6306840f1866236fa7587a8f3"
format_stamp: "Formatted at 2026-06-19 02:50:38 on dist-test-slave-9mr0"
server_key: "8985f0f273e815d12b125ad5e0a2091d"
server_key_iv: "9554b1849f3cab01b1a64f3ef2679044"
server_key_version: "encryptionkey@0"
I20260619 02:50:38.904277 22382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:38.919116 22382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:38.923321 22382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:38.923596 22382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:38.924513 22382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:50:38.925776 22382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:38.925855 22382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:38.925974 22382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:38.926028 22382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:38.954836 22382 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:33641
I20260619 02:50:38.954893 22509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:33641 every 8 connection(s)
I20260619 02:50:38.956067 22382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:50:38.958041 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 22382
I20260619 02:50:38.958184 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance
I20260619 02:50:38.958693 18968 external_mini_cluster.cc:1428] Setting key a3afdad859c23ffb013870ffca882337
Jun 19 02:50:38 dist-test-slave-9mr0 krb5kdc[22009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837438, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:38.975978 22245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:38.958494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:42815 (local address 127.18.134.62:35417)
0619 02:50:38.958721 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:38.958728 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:38.959995 (+ 1267us) server_negotiation.cc:407] Connection header received
0619 02:50:38.961765 (+ 1770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:38.961769 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:38.961831 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:38.961908 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:38.963474 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:38.964162 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:38.965626 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:38.966004 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:38.966091 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:38.969764 (+ 3673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:38.969769 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:38.969772 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:38.969806 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:38.972654 (+ 2848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:38.973434 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:38.973442 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:38.973446 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:38.973615 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:38.974122 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:38.974128 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:38.974133 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:38.974428 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:38.974578 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:38.975357 (+ 779us) server_negotiation.cc:299] Negotiation successful
0619 02:50:38.975588 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 02:50:38.976984 22512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:38.959053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35417 (local address 127.18.134.3:42815)
0619 02:50:38.959832 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:38.959882 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:38.961510 (+ 1628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:38.962160 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:38.962169 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:38.962756 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:38.963332 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:38.963350 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:38.964289 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:38.964293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:38.965520 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:38.965531 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:38.966215 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:38.966221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:38.966443 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:38.967194 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:38.967219 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:38.969528 (+ 2309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:38.972805 (+ 3277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:38.972812 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:38.972826 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:38.973281 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:38.973732 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:38.973736 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:38.973739 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:38.973949 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:38.974564 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:38.974577 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:38.974959 (+ 382us) client_negotiation.cc:769] Sending connection context
0619 02:50:38.975234 (+ 275us) client_negotiation.cc:240] Negotiation successful
0619 02:50:38.975764 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":302,"threads_started":1}
I20260619 02:50:38.978364 22510 heartbeater.cc:347] Connected to a master server at 127.18.134.62:35417
I20260619 02:50:38.978652 22510 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:38.979341 22510 heartbeater.cc:511] Master 127.18.134.62:35417 requested a full tablet report, sending...
I20260619 02:50:38.980674 22054 ts_manager.cc:194] Registered new tserver with Master: c31832b6306840f1866236fa7587a8f3 (127.18.134.3:33641)
I20260619 02:50:38.981464 22054 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.3@KRBTEST.COM'} at 127.18.134.3:42815
I20260619 02:50:38.985670 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 02:50:39 dist-test-slave-9mr0 krb5kdc[22009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837439, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 19 02:50:39 dist-test-slave-9mr0 krb5kdc[22009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837439, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:39.030025 22245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:39.016472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38344 (local address 127.18.134.62:35417)
0619 02:50:39.016701 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:39.016708 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:39.017074 (+ 366us) server_negotiation.cc:407] Connection header received
0619 02:50:39.017246 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:39.017251 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:39.017318 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:39.017403 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:39.018685 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:39.019422 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:39.020718 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:39.021360 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:39.021492 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:39.025153 (+ 3661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:39.025160 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:39.025164 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:39.025203 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:39.027842 (+ 2639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:39.028409 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:39.028412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:39.028414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:39.028546 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:39.028886 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:39.028891 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:39.028893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:39.029229 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:39.029405 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:39.029589 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 02:50:39.029748 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260619 02:50:39.030537 22054 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38344
I20260619 02:50:39.031919 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 22115
Jun 19 02:50:39 dist-test-slave-9mr0 krb5kdc[22009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837439, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.134.1@KRBTEST.COM
I20260619 02:50:39.046948 22521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:39.032821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55066 (local address 127.18.134.1:37895)
0619 02:50:39.033351 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:39.033359 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:39.033386 (+ 27us) server_negotiation.cc:407] Connection header received
0619 02:50:39.033434 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:39.033440 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:39.033637 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:39.033770 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:39.034937 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:39.035910 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:39.037107 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:39.037652 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:39.037759 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:39.041443 (+ 3684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:39.041454 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:39.041467 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:39.041501 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:39.044499 (+ 2998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:39.045175 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:39.045184 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:39.045188 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:39.045347 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:39.045647 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:39.045650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:39.045652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:39.045975 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:39.046225 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:39.046372 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 02:50:39.046562 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":226,"threads_started":1}
W20260619 02:50:39.048177 22238 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:55066
W20260619 02:50:39.049264 18968 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 22115: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 02:50:39.049454 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 22115
I20260619 02:50:39.064149 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 22249
Jun 19 02:50:39 dist-test-slave-9mr0 krb5kdc[22009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837439, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.134.2@KRBTEST.COM
I20260619 02:50:39.079545 22522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:39.065005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48604 (local address 127.18.134.2:42997)
0619 02:50:39.065534 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:39.065542 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:39.065568 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:50:39.065634 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:39.065640 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:39.065861 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:39.066011 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:39.067212 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:39.068341 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:39.069724 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:39.070282 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:39.070386 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:39.074283 (+ 3897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:39.074302 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:39.074321 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:39.074357 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:39.077120 (+ 2763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:39.077715 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:39.077722 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:39.077725 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:39.077847 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:39.078127 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:39.078130 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:39.078132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:39.078466 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:39.078727 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:39.078995 (+ 268us) server_negotiation.cc:299] Negotiation successful
0619 02:50:39.079194 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":276,"threads_started":1}
W20260619 02:50:39.080771 22372 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:48604
W20260619 02:50:39.081847 18968 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 22249: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 02:50:39.081975 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 22249
I20260619 02:50:39.093405 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 22382
Jun 19 02:50:39 dist-test-slave-9mr0 krb5kdc[22009](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837439, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.134.3@KRBTEST.COM
I20260619 02:50:39.109303 22523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:39.094218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38774 (local address 127.18.134.3:33641)
0619 02:50:39.094818 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:39.094824 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:39.094852 (+ 28us) server_negotiation.cc:407] Connection header received
0619 02:50:39.094907 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:39.094913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:39.095176 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:39.095324 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:39.096486 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:39.097569 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:39.098854 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:39.099430 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:39.099538 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:39.103596 (+ 4058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:39.103612 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:39.103631 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:39.103675 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:39.106500 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:39.107233 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:39.107241 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:39.107243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:39.107382 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:39.107758 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:39.107763 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:39.107777 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:39.108192 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:39.108426 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:39.108672 (+ 246us) server_negotiation.cc:299] Negotiation successful
0619 02:50:39.108914 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":261,"threads_started":1}
W20260619 02:50:39.110607 22505 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38774
W20260619 02:50:39.111881 18968 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 22382: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 02:50:39.112041 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 22382
I20260619 02:50:39.123867 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 22022
I20260619 02:50:39.125046 22084 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:39.223440 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 22022
2026-06-19T02:50:39Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3336 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:50:39 dist-test-slave-9mr0 krb5kdc[22527](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:50:39 dist-test-slave-9mr0 krb5kdc[22527](info): set up 2 sockets
Jun 19 02:50:39 dist-test-slave-9mr0 krb5kdc[22527](info): commencing operation
krb5kdc: starting...
W20260619 02:50:40.218672 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.959s user 0.004s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:50:40 dist-test-slave-9mr0 krb5kdc[22527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:50:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:50:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:50:40.398906 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:46803
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:45719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:46803
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:40.708822 22540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:40.709120 22540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:40.709208 22540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:40.716759 22540 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:50:40.716887 22540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:40.716933 22540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:40.716995 22540 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:50:40.717039 22540 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:50:40.727113 22540 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:45719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:46803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:46803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:40.728426 22540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:40.729663 22540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:40.741546 22548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:40.741547 22546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:40.741593 22545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:40.742255 22540 server_base.cc:1034] running on GCE node
I20260619 02:50:40.743098 22540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:40.744491 22540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:40.745733 22540 hybrid_clock.cc:648] HybridClock initialized: now 1781837440745675 us; error 37 us; skew 500 ppm
I20260619 02:50:40.746068 22540 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:40 dist-test-slave-9mr0 krb5kdc[22527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837440, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:40.749110 22540 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:50:40.750597 22540 webserver.cc:460] Webserver started at http://127.18.134.62:33609/ using document root <none> and password file <none>
I20260619 02:50:40.751281 22540 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:40.751377 22540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:40.751622 22540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:40.754552 22540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "7a1ecaed4440474ea84215a8cbd3923e"
format_stamp: "Formatted at 2026-06-19 02:50:40 on dist-test-slave-9mr0"
server_key: "25d76555ca446a12b5156d91f3003c20"
server_key_iv: "5b42ce7fc387c8336148ac9ba60b86c1"
server_key_version: "encryptionkey@0"
I20260619 02:50:40.755321 22540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "7a1ecaed4440474ea84215a8cbd3923e"
format_stamp: "Formatted at 2026-06-19 02:50:40 on dist-test-slave-9mr0"
server_key: "25d76555ca446a12b5156d91f3003c20"
server_key_iv: "5b42ce7fc387c8336148ac9ba60b86c1"
server_key_version: "encryptionkey@0"
I20260619 02:50:40.760413 22540 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:50:40.764110 22555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:40.765491 22540 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:50:40.765637 22540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "7a1ecaed4440474ea84215a8cbd3923e"
format_stamp: "Formatted at 2026-06-19 02:50:40 on dist-test-slave-9mr0"
server_key: "25d76555ca446a12b5156d91f3003c20"
server_key_iv: "5b42ce7fc387c8336148ac9ba60b86c1"
server_key_version: "encryptionkey@0"
I20260619 02:50:40.765805 22540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:40.785344 22540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:40.789418 22540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:40.789649 22540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:40.806408 22540 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:46803
I20260619 02:50:40.806427 22606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:46803 every 8 connection(s)
I20260619 02:50:40.807751 22540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:50:40.809206 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 22540
I20260619 02:50:40.809427 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:50:40.810174 18968 external_mini_cluster.cc:1428] Setting key 0ffd4f7fe06e40389f3f47bbd92a160a
I20260619 02:50:40.812562 22607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 19 02:50:40 dist-test-slave-9mr0 krb5kdc[22527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:40.824855 22607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a1ecaed4440474ea84215a8cbd3923e: Bootstrap starting.
I20260619 02:50:40.828001 22607 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a1ecaed4440474ea84215a8cbd3923e: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:40.828971 22607 log.cc:824] T 00000000000000000000000000000000 P 7a1ecaed4440474ea84215a8cbd3923e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:40.831730 22607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a1ecaed4440474ea84215a8cbd3923e: No bootstrap required, opened a new log
I20260619 02:50:40.832238 22610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:40.812186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57608 (local address 127.18.134.62:46803)
0619 02:50:40.812905 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:40.812917 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:40.812957 (+ 40us) server_negotiation.cc:407] Connection header received
0619 02:50:40.814519 (+ 1562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:40.814540 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:40.814972 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:40.815303 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:40.817000 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:40.818349 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:40.820002 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:40.820644 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:40.820776 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:40.825113 (+ 4337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:40.825121 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:40.825136 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:40.825176 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:40.827942 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:40.828787 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:40.828794 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:40.828806 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:40.829006 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:40.829388 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:40.829398 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:40.829401 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:40.830010 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:40.830305 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:40.830777 (+ 472us) server_negotiation.cc:299] Negotiation successful
0619 02:50:40.831179 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":324,"threads_started":1}
I20260619 02:50:40.838987 22607 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a1ecaed4440474ea84215a8cbd3923e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a1ecaed4440474ea84215a8cbd3923e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 46803 } }
I20260619 02:50:40.839334 22607 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a1ecaed4440474ea84215a8cbd3923e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:40.839435 22607 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a1ecaed4440474ea84215a8cbd3923e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a1ecaed4440474ea84215a8cbd3923e, State: Initialized, Role: FOLLOWER
I20260619 02:50:40.839982 22607 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a1ecaed4440474ea84215a8cbd3923e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a1ecaed4440474ea84215a8cbd3923e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 46803 } }
I20260619 02:50:40.840193 22607 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a1ecaed4440474ea84215a8cbd3923e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:50:40.840312 22607 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a1ecaed4440474ea84215a8cbd3923e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:50:40.840461 22607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a1ecaed4440474ea84215a8cbd3923e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:40.842868 22607 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a1ecaed4440474ea84215a8cbd3923e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a1ecaed4440474ea84215a8cbd3923e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 46803 } }
I20260619 02:50:40.843323 22607 leader_election.cc:302] T 00000000000000000000000000000000 P 7a1ecaed4440474ea84215a8cbd3923e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a1ecaed4440474ea84215a8cbd3923e; no voters:
I20260619 02:50:40.843861 22607 leader_election.cc:288] T 00000000000000000000000000000000 P 7a1ecaed4440474ea84215a8cbd3923e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:50:40.843977 22612 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a1ecaed4440474ea84215a8cbd3923e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:50:40.844703 22612 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a1ecaed4440474ea84215a8cbd3923e [term 1 LEADER]: Becoming Leader. State: Replica: 7a1ecaed4440474ea84215a8cbd3923e, State: Running, Role: LEADER
I20260619 02:50:40.845170 22612 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a1ecaed4440474ea84215a8cbd3923e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a1ecaed4440474ea84215a8cbd3923e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 46803 } }
I20260619 02:50:40.845855 22607 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a1ecaed4440474ea84215a8cbd3923e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:50:40.847862 22614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a1ecaed4440474ea84215a8cbd3923e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a1ecaed4440474ea84215a8cbd3923e. Latest consensus state: current_term: 1 leader_uuid: "7a1ecaed4440474ea84215a8cbd3923e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a1ecaed4440474ea84215a8cbd3923e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 46803 } } }
I20260619 02:50:40.847950 22613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a1ecaed4440474ea84215a8cbd3923e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a1ecaed4440474ea84215a8cbd3923e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a1ecaed4440474ea84215a8cbd3923e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 46803 } } }
I20260619 02:50:40.848311 22613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a1ecaed4440474ea84215a8cbd3923e [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:40.848309 22614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a1ecaed4440474ea84215a8cbd3923e [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:40.850355 22618 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:50:40.856532 22618 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:50:40.864795 22618 catalog_manager.cc:1347] Generated new cluster ID: 1a9eb400bb4e4bcca81dec9519ca66d4
I20260619 02:50:40.864916 22618 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:50:40.876449 22618 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:50:40.877709 22618 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:50:40.900370 22618 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a1ecaed4440474ea84215a8cbd3923e: Generated new TSK 0
I20260619 02:50:40.901223 22618 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.134.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
I20260619 02:50:40.955188 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:0
--local_ip_for_outbound_sockets=127.18.134.1
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:46803
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:45719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:41.267436 22633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:41.267725 22633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:41.267863 22633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:41.275019 22633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:41.275169 22633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:41.275321 22633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:50:41.285225 22633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:45719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:46803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:41.286482 22633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:41.287755 22633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:41.300040 22638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:41.300071 22641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:41.300143 22639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:41.300608 22633 server_base.cc:1034] running on GCE node
I20260619 02:50:41.301283 22633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:41.302145 22633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:41.303320 22633 hybrid_clock.cc:648] HybridClock initialized: now 1781837441303266 us; error 48 us; skew 500 ppm
I20260619 02:50:41.303601 22633 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:41 dist-test-slave-9mr0 krb5kdc[22527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837441, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:41.306413 22633 init.cc:377] Logged in from keytab as kudu/127.18.134.1@KRBTEST.COM (short username kudu)
I20260619 02:50:41.307755 22633 webserver.cc:460] Webserver started at http://127.18.134.1:46203/ using document root <none> and password file <none>
I20260619 02:50:41.308347 22633 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:41.308439 22633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:41.308681 22633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:41.311405 22633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-0/data/instance:
uuid: "622aa900b179458887199d24506cbb7c"
format_stamp: "Formatted at 2026-06-19 02:50:41 on dist-test-slave-9mr0"
server_key: "eda746e4fa41559fadb84a7d3bcc701e"
server_key_iv: "96b1d2bd370fbc76231ed213a6086cae"
server_key_version: "encryptionkey@0"
I20260619 02:50:41.312088 22633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance:
uuid: "622aa900b179458887199d24506cbb7c"
format_stamp: "Formatted at 2026-06-19 02:50:41 on dist-test-slave-9mr0"
server_key: "eda746e4fa41559fadb84a7d3bcc701e"
server_key_iv: "96b1d2bd370fbc76231ed213a6086cae"
server_key_version: "encryptionkey@0"
I20260619 02:50:41.316968 22633 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:50:41.320219 22648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:41.321487 22633 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 02:50:41.321630 22633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "622aa900b179458887199d24506cbb7c"
format_stamp: "Formatted at 2026-06-19 02:50:41 on dist-test-slave-9mr0"
server_key: "eda746e4fa41559fadb84a7d3bcc701e"
server_key_iv: "96b1d2bd370fbc76231ed213a6086cae"
server_key_version: "encryptionkey@0"
I20260619 02:50:41.321815 22633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:41.337188 22633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:41.340718 22633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:41.340952 22633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:41.341813 22633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:50:41.343070 22633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:41.343142 22633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:41.343255 22633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:41.343321 22633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:41.372833 22633 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:39667
I20260619 02:50:41.372872 22760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:39667 every 8 connection(s)
I20260619 02:50:41.374119 22633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:50:41.383690 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 22633
I20260619 02:50:41.383908 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance
I20260619 02:50:41.384526 18968 external_mini_cluster.cc:1428] Setting key c78d6cced06b7fb58792605711e65a34
Jun 19 02:50:41 dist-test-slave-9mr0 krb5kdc[22527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837441, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:41.393759 22763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:41.376615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:37545 (local address 127.18.134.62:46803)
0619 02:50:41.377053 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:41.377057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:41.378039 (+ 982us) server_negotiation.cc:407] Connection header received
0619 02:50:41.379841 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:41.379846 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:41.379919 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:41.380009 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:41.381485 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:41.382189 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:41.384046 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:41.384467 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:41.384571 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:41.388420 (+ 3849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:41.388425 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:41.388428 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:41.388458 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:41.390967 (+ 2509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:41.391623 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:41.391629 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:41.391632 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:41.391771 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:41.392156 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:41.392160 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:41.392164 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:41.392484 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:41.392613 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:41.393211 (+ 598us) server_negotiation.cc:299] Negotiation successful
0619 02:50:41.393413 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":246,"threads_started":1}
I20260619 02:50:41.394563 22764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:41.377167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:46803 (local address 127.18.134.1:37545)
0619 02:50:41.377871 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:41.377910 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:41.379553 (+ 1643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:41.380252 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:41.380261 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:41.380767 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:41.381318 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:41.381334 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:41.382334 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:41.382338 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:41.383877 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:41.383893 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:41.384706 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:41.384714 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:41.384991 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:41.385681 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:41.385708 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:41.388202 (+ 2494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:41.391093 (+ 2891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:41.391099 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:41.391112 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:41.391496 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:41.391885 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:41.391888 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:41.391890 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:41.392059 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:41.392592 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:41.392598 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:41.392894 (+ 296us) client_negotiation.cc:769] Sending connection context
0619 02:50:41.393102 (+ 208us) client_negotiation.cc:240] Negotiation successful
0619 02:50:41.393471 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":251,"threads_started":1}
I20260619 02:50:41.395974 22761 heartbeater.cc:347] Connected to a master server at 127.18.134.62:46803
I20260619 02:50:41.396232 22761 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:41.396908 22761 heartbeater.cc:511] Master 127.18.134.62:46803 requested a full tablet report, sending...
I20260619 02:50:41.399175 22572 ts_manager.cc:194] Registered new tserver with Master: 622aa900b179458887199d24506cbb7c (127.18.134.1:39667)
WARNING: no policy specified for kudu/127.18.134.2@KRBTEST.COM; defaulting to no policy
I20260619 02:50:41.401335 22572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.1@KRBTEST.COM'} at 127.18.134.1:37545
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
I20260619 02:50:41.425594 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:0
--local_ip_for_outbound_sockets=127.18.134.2
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:46803
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:45719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:41.729892 22767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:41.730183 22767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:41.730304 22767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:41.737444 22767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:41.737551 22767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:41.737695 22767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:50:41.747407 22767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:45719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:46803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:41.748651 22767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:41.749886 22767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:41.762140 22772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:41.762200 22775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:41.762251 22773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:41.762869 22767 server_base.cc:1034] running on GCE node
I20260619 02:50:41.763432 22767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:41.764369 22767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:41.765587 22767 hybrid_clock.cc:648] HybridClock initialized: now 1781837441765532 us; error 44 us; skew 500 ppm
I20260619 02:50:41.765897 22767 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:41 dist-test-slave-9mr0 krb5kdc[22527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837441, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:41.768887 22767 init.cc:377] Logged in from keytab as kudu/127.18.134.2@KRBTEST.COM (short username kudu)
I20260619 02:50:41.770360 22767 webserver.cc:460] Webserver started at http://127.18.134.2:46639/ using document root <none> and password file <none>
I20260619 02:50:41.771041 22767 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:41.771160 22767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:41.771437 22767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:41.774232 22767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-1/data/instance:
uuid: "8e40127fd19d4466aed90ccde080d6b6"
format_stamp: "Formatted at 2026-06-19 02:50:41 on dist-test-slave-9mr0"
server_key: "af18f41912bf70b55b1d672857e6cd2f"
server_key_iv: "e1716847c0bc0d6390bf609b80050bcd"
server_key_version: "encryptionkey@0"
I20260619 02:50:41.774971 22767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance:
uuid: "8e40127fd19d4466aed90ccde080d6b6"
format_stamp: "Formatted at 2026-06-19 02:50:41 on dist-test-slave-9mr0"
server_key: "af18f41912bf70b55b1d672857e6cd2f"
server_key_iv: "e1716847c0bc0d6390bf609b80050bcd"
server_key_version: "encryptionkey@0"
I20260619 02:50:41.779934 22767 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 02:50:41.783475 22782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:41.784889 22767 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:50:41.785053 22767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "8e40127fd19d4466aed90ccde080d6b6"
format_stamp: "Formatted at 2026-06-19 02:50:41 on dist-test-slave-9mr0"
server_key: "af18f41912bf70b55b1d672857e6cd2f"
server_key_iv: "e1716847c0bc0d6390bf609b80050bcd"
server_key_version: "encryptionkey@0"
I20260619 02:50:41.785256 22767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:41.803272 22767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:41.807468 22767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:41.807762 22767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:41.808841 22767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:50:41.810437 22767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:41.810525 22767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:41.810662 22767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:41.810716 22767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260619 02:50:41.842236 22767 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:35697
I20260619 02:50:41.842300 22894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:35697 every 8 connection(s)
I20260619 02:50:41.843473 22767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:50:41.851665 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 22767
I20260619 02:50:41.851843 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance
I20260619 02:50:41.852248 18968 external_mini_cluster.cc:1428] Setting key 8532de3338955a9f71374d027dcce705
Jun 19 02:50:41 dist-test-slave-9mr0 krb5kdc[22527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837441, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:41.862246 22763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:41.845834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:42383 (local address 127.18.134.62:46803)
0619 02:50:41.845990 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:41.845995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:41.847236 (+ 1241us) server_negotiation.cc:407] Connection header received
0619 02:50:41.848978 (+ 1742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:41.848983 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:41.849064 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:41.849147 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:41.850587 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:41.851415 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:41.853031 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:41.853599 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:41.853721 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:41.856899 (+ 3178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:41.856905 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:41.856908 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:41.856939 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:41.859371 (+ 2432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:41.860082 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:41.860088 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:41.860092 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:41.860227 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:41.860646 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:41.860649 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:41.860651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:41.860908 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:41.861036 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:41.861706 (+ 670us) server_negotiation.cc:299] Negotiation successful
0619 02:50:41.861918 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 02:50:41.863262 22897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:41.846363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:46803 (local address 127.18.134.2:42383)
0619 02:50:41.847082 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:41.847117 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:41.848718 (+ 1601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:41.849381 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:41.849389 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:41.849887 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:41.850425 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:41.850442 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:41.851558 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:41.851561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:41.852839 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:41.852850 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:41.853802 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:41.853810 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:41.854054 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:41.854680 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:41.854710 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:41.856709 (+ 1999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:41.859522 (+ 2813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:41.859532 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:41.859549 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:41.859969 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:41.860349 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:41.860354 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:41.860358 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:41.860545 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:41.861025 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:41.861033 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:41.861365 (+ 332us) client_negotiation.cc:769] Sending connection context
0619 02:50:41.861603 (+ 238us) client_negotiation.cc:240] Negotiation successful
0619 02:50:41.862124 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":309,"threads_started":1}
I20260619 02:50:41.864890 22895 heartbeater.cc:347] Connected to a master server at 127.18.134.62:46803
I20260619 02:50:41.865165 22895 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:41.865855 22895 heartbeater.cc:511] Master 127.18.134.62:46803 requested a full tablet report, sending...
I20260619 02:50:41.867298 22572 ts_manager.cc:194] Registered new tserver with Master: 8e40127fd19d4466aed90ccde080d6b6 (127.18.134.2:35697)
WARNING: no policy specified for kudu/127.18.134.3@KRBTEST.COM; defaulting to no policy
I20260619 02:50:41.868228 22572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.2@KRBTEST.COM'} at 127.18.134.2:42383
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
I20260619 02:50:41.894235 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:0
--local_ip_for_outbound_sockets=127.18.134.3
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:46803
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:45719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:42.201539 22900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:42.201821 22900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:42.201936 22900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:42.209158 22900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:42.209262 22900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:42.209403 22900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:50:42.218889 22900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:45719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:46803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:42.220153 22900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:42.221382 22900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:42.233383 22905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:42.233320 22908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:42.233467 22906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:42.234508 22900 server_base.cc:1034] running on GCE node
I20260619 02:50:42.235126 22900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:42.236052 22900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:42.237268 22900 hybrid_clock.cc:648] HybridClock initialized: now 1781837442237216 us; error 42 us; skew 500 ppm
I20260619 02:50:42.237581 22900 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:42 dist-test-slave-9mr0 krb5kdc[22527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837442, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:42.240545 22900 init.cc:377] Logged in from keytab as kudu/127.18.134.3@KRBTEST.COM (short username kudu)
I20260619 02:50:42.241922 22900 webserver.cc:460] Webserver started at http://127.18.134.3:32949/ using document root <none> and password file <none>
I20260619 02:50:42.242548 22900 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:42.242689 22900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:42.242960 22900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:42.245707 22900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-2/data/instance:
uuid: "77d12b47997048caae6e8e14626a1af7"
format_stamp: "Formatted at 2026-06-19 02:50:42 on dist-test-slave-9mr0"
server_key: "36aa4e684d8bb711117f9c0e063ca78a"
server_key_iv: "31423ae522a386b19308e20549ba73e3"
server_key_version: "encryptionkey@0"
I20260619 02:50:42.246426 22900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance:
uuid: "77d12b47997048caae6e8e14626a1af7"
format_stamp: "Formatted at 2026-06-19 02:50:42 on dist-test-slave-9mr0"
server_key: "36aa4e684d8bb711117f9c0e063ca78a"
server_key_iv: "31423ae522a386b19308e20549ba73e3"
server_key_version: "encryptionkey@0"
I20260619 02:50:42.251304 22900 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 02:50:42.254623 22915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:42.255915 22900 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260619 02:50:42.256049 22900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "77d12b47997048caae6e8e14626a1af7"
format_stamp: "Formatted at 2026-06-19 02:50:42 on dist-test-slave-9mr0"
server_key: "36aa4e684d8bb711117f9c0e063ca78a"
server_key_iv: "31423ae522a386b19308e20549ba73e3"
server_key_version: "encryptionkey@0"
I20260619 02:50:42.256232 22900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:42.270371 22900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:42.274237 22900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:42.274489 22900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:42.275441 22900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:50:42.276678 22900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:42.276746 22900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:42.276854 22900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:42.276911 22900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:42.305738 22900 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:35733
I20260619 02:50:42.305778 23027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:35733 every 8 connection(s)
I20260619 02:50:42.307075 22900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:50:42.310930 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 22900
I20260619 02:50:42.311122 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance
I20260619 02:50:42.311496 18968 external_mini_cluster.cc:1428] Setting key 1c80644267a19d3b3b55b6242c168da0
Jun 19 02:50:42 dist-test-slave-9mr0 krb5kdc[22527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837442, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:42.325461 22763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:42.309392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:53519 (local address 127.18.134.62:46803)
0619 02:50:42.309556 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:42.309561 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:42.310784 (+ 1223us) server_negotiation.cc:407] Connection header received
0619 02:50:42.312701 (+ 1917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:42.312707 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:42.312788 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:42.312867 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:42.314417 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:42.315152 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:42.316629 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:42.317017 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:42.317121 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:42.320248 (+ 3127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:42.320252 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:42.320255 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:42.320283 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:42.322689 (+ 2406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:42.323330 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:42.323335 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:42.323339 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:42.323451 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:42.323828 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:42.323833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:42.323837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:42.324084 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:42.324209 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:42.324864 (+ 655us) server_negotiation.cc:299] Negotiation successful
0619 02:50:42.325132 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 02:50:42.326747 23030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:42.309817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:46803 (local address 127.18.134.3:53519)
0619 02:50:42.310578 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:42.310616 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:42.312441 (+ 1825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:42.313144 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:42.313156 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:42.313671 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:42.314274 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:42.314290 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:42.315299 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:42.315303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:42.316501 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:42.316512 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:42.317232 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:42.317240 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:42.317447 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:42.318042 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:42.318062 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:42.320056 (+ 1994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:42.322819 (+ 2763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:42.322827 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:42.322842 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:42.323225 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:42.323547 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:42.323552 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:42.323556 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:42.323727 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:42.324208 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:42.324216 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:42.324519 (+ 303us) client_negotiation.cc:769] Sending connection context
0619 02:50:42.324764 (+ 245us) client_negotiation.cc:240] Negotiation successful
0619 02:50:42.325328 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":310,"threads_started":1}
I20260619 02:50:42.328241 23028 heartbeater.cc:347] Connected to a master server at 127.18.134.62:46803
I20260619 02:50:42.328521 23028 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:42.329192 23028 heartbeater.cc:511] Master 127.18.134.62:46803 requested a full tablet report, sending...
I20260619 02:50:42.330468 22572 ts_manager.cc:194] Registered new tserver with Master: 77d12b47997048caae6e8e14626a1af7 (127.18.134.3:35733)
I20260619 02:50:42.331239 22572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.3@KRBTEST.COM'} at 127.18.134.3:53519
I20260619 02:50:42.338697 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 02:50:42 dist-test-slave-9mr0 krb5kdc[22527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 02:50:42 dist-test-slave-9mr0 krb5kdc[22527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.1@KRBTEST.COM
I20260619 02:50:42.381027 23035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:42.367100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58972 (local address 127.18.134.1:39667)
0619 02:50:42.367718 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:42.367724 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:42.367755 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:50:42.367911 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:42.367919 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:42.368132 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:42.368260 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:42.369549 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:42.370623 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:42.371813 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:42.372374 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:42.372489 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:42.376200 (+ 3711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:42.376209 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:42.376220 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:42.376247 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:42.378884 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:42.379428 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:42.379434 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:42.379436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:42.379542 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:42.379850 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:42.379853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:42.379855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:42.380137 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:42.380334 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:42.380508 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 02:50:42.380690 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":369,"threads_started":1}
Jun 19 02:50:42 dist-test-slave-9mr0 krb5kdc[22527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:42.396543 22763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:42.384624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57612 (local address 127.18.134.62:46803)
0619 02:50:42.384818 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:42.384824 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:42.384974 (+ 150us) server_negotiation.cc:407] Connection header received
0619 02:50:42.385145 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:42.385150 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:42.385209 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:42.385285 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:42.386381 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:42.387125 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:42.388290 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:42.388734 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:42.388840 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:42.392376 (+ 3536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:42.392381 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:42.392384 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:42.392415 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:42.394517 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:42.395094 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:42.395098 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:42.395100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:42.395214 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:42.395458 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:42.395461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:42.395463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:42.395757 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:42.395917 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:42.396126 (+ 209us) server_negotiation.cc:299] Negotiation successful
0619 02:50:42.396270 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260619 02:50:42.397157 22572 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57612
I20260619 02:50:42.398016 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 22633
I20260619 02:50:42.404515 22761 heartbeater.cc:503] Master 127.18.134.62:46803 was elected leader, sending a full tablet report...
I20260619 02:50:42.413425 23035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:42.398853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58982 (local address 127.18.134.1:39667)
0619 02:50:42.399033 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:42.399039 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:42.399249 (+ 210us) server_negotiation.cc:407] Connection header received
0619 02:50:42.399359 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:42.399363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:42.399421 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:42.399493 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:42.400575 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:42.401244 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:42.402382 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:42.402820 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:42.402915 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:42.404968 (+ 2053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:42.404974 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:42.404978 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:42.405014 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:42.411270 (+ 6256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:42.411901 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:42.411905 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:42.411908 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:42.412019 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:42.412311 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:42.412314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:42.412316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:42.412551 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:42.412712 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:42.412959 (+ 247us) server_negotiation.cc:299] Negotiation successful
0619 02:50:42.413138 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 02:50:42.413919 22756 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:42.493533 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 22633
I20260619 02:50:42.505761 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 22767
Jun 19 02:50:42 dist-test-slave-9mr0 krb5kdc[22527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.2@KRBTEST.COM
I20260619 02:50:42.520152 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:42.506579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38518 (local address 127.18.134.2:35697)
0619 02:50:42.507067 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:42.507073 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:42.507097 (+ 24us) server_negotiation.cc:407] Connection header received
0619 02:50:42.507154 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:42.507159 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:42.507371 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:42.507499 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:42.508583 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:42.509551 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:42.510793 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:42.511336 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:42.511439 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:42.515235 (+ 3796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:42.515246 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:42.515260 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:42.515294 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:42.517841 (+ 2547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:42.518426 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:42.518433 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:42.518435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:42.518546 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:42.518848 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:42.518851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:42.518853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:42.519161 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:42.519388 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:42.519600 (+ 212us) server_negotiation.cc:299] Negotiation successful
0619 02:50:42.519800 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":221,"threads_started":1}
I20260619 02:50:42.521270 22890 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:42.605515 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 22767
I20260619 02:50:42.617622 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 22900
Jun 19 02:50:42 dist-test-slave-9mr0 krb5kdc[22527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.3@KRBTEST.COM
I20260619 02:50:42.633150 23042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:42.618553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49836 (local address 127.18.134.3:35733)
0619 02:50:42.619217 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:42.619224 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:42.619251 (+ 27us) server_negotiation.cc:407] Connection header received
0619 02:50:42.619333 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:42.619340 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:42.619547 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:42.619687 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:42.620858 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:42.621871 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:42.623347 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:42.623971 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:42.624092 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:42.628124 (+ 4032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:42.628135 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:42.628149 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:42.628182 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:42.630914 (+ 2732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:42.631472 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:42.631478 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:42.631480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:42.631603 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:42.631874 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:42.631878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:42.631880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:42.632192 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:42.632404 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:42.632616 (+ 212us) server_negotiation.cc:299] Negotiation successful
0619 02:50:42.632806 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":366,"threads_started":1}
I20260619 02:50:42.634392 23023 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:42.714864 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 22900
I20260619 02:50:42.726454 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 22540
I20260619 02:50:42.727656 22602 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:42.826560 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 22540
2026-06-19T02:50:42Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3603 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:50:42 dist-test-slave-9mr0 krb5kdc[23047](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:50:42 dist-test-slave-9mr0 krb5kdc[23047](info): set up 2 sockets
Jun 19 02:50:42 dist-test-slave-9mr0 krb5kdc[23047](info): commencing operation
krb5kdc: starting...
W20260619 02:50:43.824136 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.960s user 0.002s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:50:43 dist-test-slave-9mr0 krb5kdc[23047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:50:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:50:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:50:44.004266 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:34835
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:43383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:34835
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:44.308194 23060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:44.308490 23060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:44.308626 23060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:44.315742 23060 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:50:44.315866 23060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:44.315946 23060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:44.316018 23060 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:50:44.316082 23060 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:50:44.325809 23060 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:43383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:34835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:34835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:44.327138 23060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:44.328368 23060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:44.339191 23065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:44.339190 23066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:44.339262 23068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:44.340065 23060 server_base.cc:1034] running on GCE node
I20260619 02:50:44.340814 23060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:44.342214 23060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:44.343405 23060 hybrid_clock.cc:648] HybridClock initialized: now 1781837444343355 us; error 46 us; skew 500 ppm
I20260619 02:50:44.343717 23060 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:44 dist-test-slave-9mr0 krb5kdc[23047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837444, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:44.346527 23060 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:50:44.347600 23060 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:44.347687 23060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:44.347944 23060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:44.350754 23060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "670600acaf6f480dbbe3cb8e55874b42"
format_stamp: "Formatted at 2026-06-19 02:50:44 on dist-test-slave-9mr0"
server_key: "74fc791948e33c4a19c5cdb28ec1f214"
server_key_iv: "585af93ec8d4391bacd0df3a830bdb08"
server_key_version: "encryptionkey@0"
I20260619 02:50:44.351470 23060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "670600acaf6f480dbbe3cb8e55874b42"
format_stamp: "Formatted at 2026-06-19 02:50:44 on dist-test-slave-9mr0"
server_key: "74fc791948e33c4a19c5cdb28ec1f214"
server_key_iv: "585af93ec8d4391bacd0df3a830bdb08"
server_key_version: "encryptionkey@0"
I20260619 02:50:44.356320 23060 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:50:44.359625 23074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:44.360977 23060 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:50:44.361177 23060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "670600acaf6f480dbbe3cb8e55874b42"
format_stamp: "Formatted at 2026-06-19 02:50:44 on dist-test-slave-9mr0"
server_key: "74fc791948e33c4a19c5cdb28ec1f214"
server_key_iv: "585af93ec8d4391bacd0df3a830bdb08"
server_key_version: "encryptionkey@0"
I20260619 02:50:44.361368 23060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:44.382591 23060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:44.386767 23060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:44.387003 23060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:44.403956 23060 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:34835
I20260619 02:50:44.403949 23125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:34835 every 8 connection(s)
I20260619 02:50:44.404806 23060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:50:44.408586 23126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:44.410701 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 23060
I20260619 02:50:44.410916 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:50:44.411684 18968 external_mini_cluster.cc:1428] Setting key 5ed6533362c9166033efe798a4ebd83e
I20260619 02:50:44.420135 23126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 670600acaf6f480dbbe3cb8e55874b42: Bootstrap starting.
I20260619 02:50:44.423105 23126 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 670600acaf6f480dbbe3cb8e55874b42: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:44.424031 23126 log.cc:824] T 00000000000000000000000000000000 P 670600acaf6f480dbbe3cb8e55874b42: Log is configured to *not* fsync() on all Append() calls
Jun 19 02:50:44 dist-test-slave-9mr0 krb5kdc[23047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:44.427124 23126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 670600acaf6f480dbbe3cb8e55874b42: No bootstrap required, opened a new log
I20260619 02:50:44.433933 23129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:44.413847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44302 (local address 127.18.134.62:34835)
0619 02:50:44.414658 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:44.414673 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:44.414723 (+ 50us) server_negotiation.cc:407] Connection header received
0619 02:50:44.416286 (+ 1563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:44.416309 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:44.416702 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:44.417045 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:44.418429 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:44.419726 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:44.421040 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:44.421716 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:44.421892 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:44.425735 (+ 3843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:44.425746 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:44.425763 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:44.425809 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:44.429439 (+ 3630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:44.430078 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:44.430086 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:44.430095 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:44.430300 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:44.430683 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:44.430688 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:44.430691 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:44.431376 (+ 685us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:44.431636 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:44.432104 (+ 468us) server_negotiation.cc:299] Negotiation successful
0619 02:50:44.432554 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":361,"threads_started":1}
I20260619 02:50:44.436800 23126 raft_consensus.cc:348] T 00000000000000000000000000000000 P 670600acaf6f480dbbe3cb8e55874b42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "670600acaf6f480dbbe3cb8e55874b42" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 34835 } }
I20260619 02:50:44.437165 23126 raft_consensus.cc:374] T 00000000000000000000000000000000 P 670600acaf6f480dbbe3cb8e55874b42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:44.437242 23126 raft_consensus.cc:729] T 00000000000000000000000000000000 P 670600acaf6f480dbbe3cb8e55874b42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 670600acaf6f480dbbe3cb8e55874b42, State: Initialized, Role: FOLLOWER
I20260619 02:50:44.437696 23126 consensus_queue.cc:260] T 00000000000000000000000000000000 P 670600acaf6f480dbbe3cb8e55874b42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "670600acaf6f480dbbe3cb8e55874b42" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 34835 } }
I20260619 02:50:44.437879 23126 raft_consensus.cc:388] T 00000000000000000000000000000000 P 670600acaf6f480dbbe3cb8e55874b42 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:50:44.437978 23126 raft_consensus.cc:482] T 00000000000000000000000000000000 P 670600acaf6f480dbbe3cb8e55874b42 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:50:44.438107 23126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 670600acaf6f480dbbe3cb8e55874b42 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:44.440392 23126 raft_consensus.cc:504] T 00000000000000000000000000000000 P 670600acaf6f480dbbe3cb8e55874b42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "670600acaf6f480dbbe3cb8e55874b42" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 34835 } }
I20260619 02:50:44.440855 23126 leader_election.cc:302] T 00000000000000000000000000000000 P 670600acaf6f480dbbe3cb8e55874b42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 670600acaf6f480dbbe3cb8e55874b42; no voters:
I20260619 02:50:44.441363 23126 leader_election.cc:288] T 00000000000000000000000000000000 P 670600acaf6f480dbbe3cb8e55874b42 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:50:44.441483 23131 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 670600acaf6f480dbbe3cb8e55874b42 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:50:44.442076 23131 raft_consensus.cc:686] T 00000000000000000000000000000000 P 670600acaf6f480dbbe3cb8e55874b42 [term 1 LEADER]: Becoming Leader. State: Replica: 670600acaf6f480dbbe3cb8e55874b42, State: Running, Role: LEADER
I20260619 02:50:44.442582 23131 consensus_queue.cc:237] T 00000000000000000000000000000000 P 670600acaf6f480dbbe3cb8e55874b42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "670600acaf6f480dbbe3cb8e55874b42" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 34835 } }
I20260619 02:50:44.443292 23126 sys_catalog.cc:564] T 00000000000000000000000000000000 P 670600acaf6f480dbbe3cb8e55874b42 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:50:44.445359 23132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 670600acaf6f480dbbe3cb8e55874b42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "670600acaf6f480dbbe3cb8e55874b42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "670600acaf6f480dbbe3cb8e55874b42" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 34835 } } }
I20260619 02:50:44.445402 23133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 670600acaf6f480dbbe3cb8e55874b42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 670600acaf6f480dbbe3cb8e55874b42. Latest consensus state: current_term: 1 leader_uuid: "670600acaf6f480dbbe3cb8e55874b42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "670600acaf6f480dbbe3cb8e55874b42" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 34835 } } }
I20260619 02:50:44.445734 23133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 670600acaf6f480dbbe3cb8e55874b42 [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:44.445739 23132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 670600acaf6f480dbbe3cb8e55874b42 [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:44.447562 23138 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:50:44.452492 23138 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:50:44.462015 23138 catalog_manager.cc:1347] Generated new cluster ID: 00f5206199d740d8bb14af74b5e22384
I20260619 02:50:44.462132 23138 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:50:44.482249 23138 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:50:44.483632 23138 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:50:44.490509 23138 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 670600acaf6f480dbbe3cb8e55874b42: Generated new TSK 0
I20260619 02:50:44.491533 23138 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.134.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
I20260619 02:50:44.545853 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:0
--local_ip_for_outbound_sockets=127.18.134.1
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:34835
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:43383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:44.857275 23152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:44.857577 23152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:44.857712 23152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:44.865006 23152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:44.865124 23152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:44.865278 23152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:50:44.875149 23152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:43383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:34835
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:44.876396 23152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:44.877652 23152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:44.890322 23158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:44.890314 23157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:44.890985 23160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:44.891038 23152 server_base.cc:1034] running on GCE node
I20260619 02:50:44.891656 23152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:44.892563 23152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:44.893772 23152 hybrid_clock.cc:648] HybridClock initialized: now 1781837444893705 us; error 59 us; skew 500 ppm
I20260619 02:50:44.894096 23152 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:44 dist-test-slave-9mr0 krb5kdc[23047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837444, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:44.897130 23152 init.cc:377] Logged in from keytab as kudu/127.18.134.1@KRBTEST.COM (short username kudu)
I20260619 02:50:44.898253 23152 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:44.898351 23152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:44.898665 23152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:44.901460 23152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-0/data/instance:
uuid: "73d263e596b94d8991c79adccd0e0f1e"
format_stamp: "Formatted at 2026-06-19 02:50:44 on dist-test-slave-9mr0"
server_key: "a2cbf87be7833455db0d3edd3293a4b1"
server_key_iv: "00dc0c3da8609706efc0119f33c8a2ab"
server_key_version: "encryptionkey@0"
I20260619 02:50:44.902222 23152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance:
uuid: "73d263e596b94d8991c79adccd0e0f1e"
format_stamp: "Formatted at 2026-06-19 02:50:44 on dist-test-slave-9mr0"
server_key: "a2cbf87be7833455db0d3edd3293a4b1"
server_key_iv: "00dc0c3da8609706efc0119f33c8a2ab"
server_key_version: "encryptionkey@0"
I20260619 02:50:44.907066 23152 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260619 02:50:44.910568 23166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:44.911909 23152 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:50:44.912091 23152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "73d263e596b94d8991c79adccd0e0f1e"
format_stamp: "Formatted at 2026-06-19 02:50:44 on dist-test-slave-9mr0"
server_key: "a2cbf87be7833455db0d3edd3293a4b1"
server_key_iv: "00dc0c3da8609706efc0119f33c8a2ab"
server_key_version: "encryptionkey@0"
I20260619 02:50:44.912292 23152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:44.939918 23152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:44.944142 23152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:44.944412 23152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:44.945385 23152 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:50:44.946827 23152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:44.946924 23152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:44.947039 23152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:44.947111 23152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:44.978186 23152 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:44077
I20260619 02:50:44.978268 23278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:44077 every 8 connection(s)
I20260619 02:50:44.979046 23152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:50:44.984623 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 23152
I20260619 02:50:44.984829 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance
I20260619 02:50:44.985488 18968 external_mini_cluster.cc:1428] Setting key 88e1d251cda91e7ff12714f718b98e9b
Jun 19 02:50:44 dist-test-slave-9mr0 krb5kdc[23047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837444, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:44.999351 23281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:44.981548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:52253 (local address 127.18.134.62:34835)
0619 02:50:44.982022 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:44.982026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:44.983160 (+ 1134us) server_negotiation.cc:407] Connection header received
0619 02:50:44.984992 (+ 1832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:44.984997 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:44.985084 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:44.985180 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:44.986853 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:44.987726 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:44.989250 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:44.989740 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:44.989859 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:44.993719 (+ 3860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:44.993724 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:44.993727 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:44.993761 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:44.996388 (+ 2627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:44.997163 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:44.997167 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:44.997170 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:44.997285 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:44.997690 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:44.997694 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:44.997696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:44.997964 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:44.998090 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:44.998795 (+ 705us) server_negotiation.cc:299] Negotiation successful
0619 02:50:44.999007 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":252,"threads_started":1}
I20260619 02:50:45.000412 23282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:44.982178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:34835 (local address 127.18.134.1:52253)
0619 02:50:44.982992 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:44.983039 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:44.984706 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:44.985430 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:44.985439 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:44.985986 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:44.986695 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:44.986715 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:44.987873 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:44.987877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:44.989118 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:44.989130 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:44.989970 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:44.989978 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:44.990198 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:44.990980 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:44.991004 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:44.993463 (+ 2459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:44.996561 (+ 3098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:44.996572 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:44.996589 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:44.997027 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:44.997391 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:44.997397 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:44.997400 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:44.997577 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:44.998084 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:44.998101 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:44.998431 (+ 330us) client_negotiation.cc:769] Sending connection context
0619 02:50:44.998710 (+ 279us) client_negotiation.cc:240] Negotiation successful
0619 02:50:44.999258 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":318,"threads_started":1}
WARNING: no policy specified for kudu/127.18.134.2@KRBTEST.COM; defaulting to no policy
I20260619 02:50:45.002135 23279 heartbeater.cc:347] Connected to a master server at 127.18.134.62:34835
I20260619 02:50:45.002409 23279 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:45.003119 23279 heartbeater.cc:511] Master 127.18.134.62:34835 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.2@KRBTEST.COM" created.
I20260619 02:50:45.005125 23091 ts_manager.cc:194] Registered new tserver with Master: 73d263e596b94d8991c79adccd0e0f1e (127.18.134.1:44077)
I20260619 02:50:45.006510 23091 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.1@KRBTEST.COM'} at 127.18.134.1:52253
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
I20260619 02:50:45.025990 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:0
--local_ip_for_outbound_sockets=127.18.134.2
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:34835
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:43383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:45.333603 23285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:45.333880 23285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:45.333976 23285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:45.341078 23285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:45.341179 23285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:45.341302 23285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:50:45.350936 23285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:43383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:34835
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:45.352118 23285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:45.353297 23285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:45.365375 23290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:45.365437 23293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:45.365514 23291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:45.366361 23285 server_base.cc:1034] running on GCE node
I20260619 02:50:45.367065 23285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:45.368063 23285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:45.369309 23285 hybrid_clock.cc:648] HybridClock initialized: now 1781837445369250 us; error 48 us; skew 500 ppm
I20260619 02:50:45.369638 23285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:45 dist-test-slave-9mr0 krb5kdc[23047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837445, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:45.372740 23285 init.cc:377] Logged in from keytab as kudu/127.18.134.2@KRBTEST.COM (short username kudu)
I20260619 02:50:45.373888 23285 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:45.373989 23285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:45.374284 23285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:45.377139 23285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-1/data/instance:
uuid: "00a18ff270ea4dc99518c40fda65f190"
format_stamp: "Formatted at 2026-06-19 02:50:45 on dist-test-slave-9mr0"
server_key: "86265c4969dcc0bc045ac9f150a33fb4"
server_key_iv: "a4902071b60a7aa51c2f800021e59fab"
server_key_version: "encryptionkey@0"
I20260619 02:50:45.377902 23285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance:
uuid: "00a18ff270ea4dc99518c40fda65f190"
format_stamp: "Formatted at 2026-06-19 02:50:45 on dist-test-slave-9mr0"
server_key: "86265c4969dcc0bc045ac9f150a33fb4"
server_key_iv: "a4902071b60a7aa51c2f800021e59fab"
server_key_version: "encryptionkey@0"
I20260619 02:50:45.382833 23285 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260619 02:50:45.386160 23299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:45.387477 23285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:50:45.387648 23285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "00a18ff270ea4dc99518c40fda65f190"
format_stamp: "Formatted at 2026-06-19 02:50:45 on dist-test-slave-9mr0"
server_key: "86265c4969dcc0bc045ac9f150a33fb4"
server_key_iv: "a4902071b60a7aa51c2f800021e59fab"
server_key_version: "encryptionkey@0"
I20260619 02:50:45.387846 23285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:45.411813 23285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:45.415921 23285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:45.416164 23285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:45.417160 23285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:50:45.418394 23285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:45.418489 23285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:45.418598 23285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:45.418690 23285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260619 02:50:45.447086 23285 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:44277
I20260619 02:50:45.447177 23411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:44277 every 8 connection(s)
I20260619 02:50:45.447902 23285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:50:45.452548 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 23285
I20260619 02:50:45.452737 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance
I20260619 02:50:45.453266 18968 external_mini_cluster.cc:1428] Setting key ac0c766343f6ea962e70e3db7a89159e
Jun 19 02:50:45 dist-test-slave-9mr0 krb5kdc[23047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837445, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:45.468458 23281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:45.450095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:36197 (local address 127.18.134.62:34835)
0619 02:50:45.450258 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:45.450263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:45.451346 (+ 1083us) server_negotiation.cc:407] Connection header received
0619 02:50:45.453394 (+ 2048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:45.453398 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:45.453467 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:45.453564 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:45.455267 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:45.455952 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:45.458033 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:45.458481 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:45.458583 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:45.462337 (+ 3754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:45.462343 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:45.462348 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:45.462384 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:45.465383 (+ 2999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:45.466084 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:45.466089 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:45.466092 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:45.466247 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:45.466749 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:45.466754 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:45.466758 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:45.467082 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:45.467223 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:45.467892 (+ 669us) server_negotiation.cc:299] Negotiation successful
0619 02:50:45.468116 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 02:50:45.469339 23414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:45.450497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:34835 (local address 127.18.134.2:36197)
0619 02:50:45.451197 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:45.451233 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:45.453103 (+ 1870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:45.453811 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:45.453822 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:45.454387 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:45.455107 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:45.455127 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:45.456113 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:45.456119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:45.457904 (+ 1785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:45.457917 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:45.458711 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:45.458717 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:45.458959 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:45.459613 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:45.459650 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:45.462130 (+ 2480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:45.465502 (+ 3372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:45.465509 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:45.465527 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:45.465971 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:45.466385 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:45.466390 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:45.466392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:45.466614 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:45.467203 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:45.467209 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:45.467542 (+ 333us) client_negotiation.cc:769] Sending connection context
0619 02:50:45.467768 (+ 226us) client_negotiation.cc:240] Negotiation successful
0619 02:50:45.468229 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":289,"threads_started":1}
I20260619 02:50:45.470800 23412 heartbeater.cc:347] Connected to a master server at 127.18.134.62:34835
I20260619 02:50:45.471127 23412 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.18.134.3@KRBTEST.COM; defaulting to no policy
I20260619 02:50:45.471899 23412 heartbeater.cc:511] Master 127.18.134.62:34835 requested a full tablet report, sending...
I20260619 02:50:45.473433 23091 ts_manager.cc:194] Registered new tserver with Master: 00a18ff270ea4dc99518c40fda65f190 (127.18.134.2:44277)
I20260619 02:50:45.474399 23091 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.2@KRBTEST.COM'} at 127.18.134.2:36197
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
I20260619 02:50:45.497809 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:0
--local_ip_for_outbound_sockets=127.18.134.3
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:34835
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:43383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:45.798868 23417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:45.799150 23417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:45.799283 23417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:45.806363 23417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:45.806465 23417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:45.806612 23417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:50:45.816155 23417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:43383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:34835
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:45.817445 23417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:45.818750 23417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:45.830823 23422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:45.830920 23423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:45.830920 23425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:45.831902 23417 server_base.cc:1034] running on GCE node
I20260619 02:50:45.832532 23417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:45.833506 23417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:45.834722 23417 hybrid_clock.cc:648] HybridClock initialized: now 1781837445834665 us; error 48 us; skew 500 ppm
I20260619 02:50:45.835022 23417 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:45 dist-test-slave-9mr0 krb5kdc[23047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837445, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:45.837924 23417 init.cc:377] Logged in from keytab as kudu/127.18.134.3@KRBTEST.COM (short username kudu)
I20260619 02:50:45.839012 23417 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:45.839121 23417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:45.839393 23417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:45.842123 23417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-2/data/instance:
uuid: "e6272816ff2c4b29b86ae64c1f6da25b"
format_stamp: "Formatted at 2026-06-19 02:50:45 on dist-test-slave-9mr0"
server_key: "040f4af0d93d16c790b59f24da8eb3f6"
server_key_iv: "bd32a441be0a43d925ba7420aadb174b"
server_key_version: "encryptionkey@0"
I20260619 02:50:45.842896 23417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance:
uuid: "e6272816ff2c4b29b86ae64c1f6da25b"
format_stamp: "Formatted at 2026-06-19 02:50:45 on dist-test-slave-9mr0"
server_key: "040f4af0d93d16c790b59f24da8eb3f6"
server_key_iv: "bd32a441be0a43d925ba7420aadb174b"
server_key_version: "encryptionkey@0"
I20260619 02:50:45.847841 23417 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260619 02:50:45.851155 23431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:45.852423 23417 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 02:50:45.852566 23417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "e6272816ff2c4b29b86ae64c1f6da25b"
format_stamp: "Formatted at 2026-06-19 02:50:45 on dist-test-slave-9mr0"
server_key: "040f4af0d93d16c790b59f24da8eb3f6"
server_key_iv: "bd32a441be0a43d925ba7420aadb174b"
server_key_version: "encryptionkey@0"
I20260619 02:50:45.852756 23417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:45.870759 23417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:45.874812 23417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:45.875047 23417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:45.875959 23417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:50:45.877220 23417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:45.877318 23417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:45.877429 23417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:45.877497 23417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:45.905994 23417 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:33559
I20260619 02:50:45.906047 23543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:33559 every 8 connection(s)
I20260619 02:50:45.906839 23417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:50:45.913818 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 23417
I20260619 02:50:45.913992 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance
I20260619 02:50:45.914544 18968 external_mini_cluster.cc:1428] Setting key 2e2560daf3173cedba9fb50ef0a499dc
Jun 19 02:50:45 dist-test-slave-9mr0 krb5kdc[23047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837445, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:45.925691 23281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:45.909126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:47945 (local address 127.18.134.62:34835)
0619 02:50:45.909311 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:45.909318 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:45.910576 (+ 1258us) server_negotiation.cc:407] Connection header received
0619 02:50:45.912347 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:45.912351 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:45.912432 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:45.912529 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:45.914162 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:45.915186 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:45.916846 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:45.917313 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:45.917420 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:45.920630 (+ 3210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:45.920635 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:45.920639 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:45.920671 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:45.922973 (+ 2302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:45.923624 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:45.923629 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:45.923631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:45.923738 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:45.924116 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:45.924121 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:45.924125 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:45.924377 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:45.924511 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:45.925145 (+ 634us) server_negotiation.cc:299] Negotiation successful
0619 02:50:45.925346 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 02:50:45.926618 23546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:45.909681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:34835 (local address 127.18.134.3:47945)
0619 02:50:45.910404 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:45.910452 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:45.912081 (+ 1629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:45.912792 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:45.912800 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:45.913355 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:45.913979 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:45.914003 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:45.915336 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:45.915340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:45.916688 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:45.916703 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:45.917519 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:45.917526 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:45.917748 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:45.918321 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:45.918343 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:45.920432 (+ 2089us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:45.923098 (+ 2666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:45.923107 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:45.923125 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:45.923518 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:45.923834 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:45.923839 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:45.923843 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:45.924018 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:45.924499 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:45.924507 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:45.924821 (+ 314us) client_negotiation.cc:769] Sending connection context
0619 02:50:45.925060 (+ 239us) client_negotiation.cc:240] Negotiation successful
0619 02:50:45.925572 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":306,"threads_started":1}
I20260619 02:50:45.927970 23544 heartbeater.cc:347] Connected to a master server at 127.18.134.62:34835
I20260619 02:50:45.928299 23544 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:45.928928 23544 heartbeater.cc:511] Master 127.18.134.62:34835 requested a full tablet report, sending...
I20260619 02:50:45.930330 23091 ts_manager.cc:194] Registered new tserver with Master: e6272816ff2c4b29b86ae64c1f6da25b (127.18.134.3:33559)
I20260619 02:50:45.931150 23091 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.3@KRBTEST.COM'} at 127.18.134.3:47945
I20260619 02:50:45.942202 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:50:45.958699 23281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:45.946608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44312 (local address 127.18.134.62:34835)
0619 02:50:45.946780 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:45.946785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:45.947243 (+ 458us) server_negotiation.cc:407] Connection header received
0619 02:50:45.947404 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:45.947408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:45.947472 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:45.947559 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:45.948887 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:45.949625 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:45.950883 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:45.951333 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:45.951438 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:45.953590 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:45.953594 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:45.953598 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:45.953629 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:45.956256 (+ 2627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:45.956874 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:45.956879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:45.956883 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:45.957017 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:45.957309 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:45.957312 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:45.957314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:45.957620 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:45.957783 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:45.958209 (+ 426us) server_negotiation.cc:299] Negotiation successful
0619 02:50:45.958403 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 02:50:45.962812 23091 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44312:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:50:45.969368 23091 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260619 02:50:45.983618 23275 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.18.134.1:44077
I20260619 02:50:46.003576 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:45.980831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:44077 (local address 127.0.0.1:57284)
0619 02:50:45.993827 (+ 12996us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:45.993890 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:45.994142 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:45.995275 (+ 1133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:45.995283 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:45.995318 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:45.995710 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:45.995723 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:45.997638 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:45.997643 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:46.000102 (+ 2459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:46.000116 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:46.002091 (+ 1975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:46.002101 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:46.002728 (+ 627us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:46.002798 (+ 70us) client_negotiation.cc:769] Sending connection context
0619 02:50:46.002925 (+ 127us) client_negotiation.cc:240] Negotiation successful
0619 02:50:46.003108 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12694,"mutex_wait_us":32,"thread_start_us":16417,"threads_started":1}
I20260619 02:50:46.003609 23558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:45.980936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57284 (local address 127.18.134.1:44077)
0619 02:50:45.993123 (+ 12187us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:45.993134 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:45.994355 (+ 1221us) server_negotiation.cc:407] Connection header received
0619 02:50:45.994467 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:45.994476 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:45.994800 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:45.994983 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:45.995929 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:45.997457 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:46.000258 (+ 2801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:46.001674 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:46.001797 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:46.002033 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:46.003137 (+ 1104us) server_negotiation.cc:299] Negotiation successful
0619 02:50:46.003301 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11969,"thread_start_us":12216,"threads_started":1}
I20260619 02:50:46.005102 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:45.980877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:44277 (local address 127.0.0.1:44742)
0619 02:50:45.997509 (+ 16632us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:45.997526 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:45.997642 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:45.998500 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:45.998521 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:45.998547 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:45.999055 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:45.999067 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:46.000806 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:46.000811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:46.002435 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:46.002448 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:46.004371 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:46.004377 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:46.004525 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:46.004554 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 02:50:46.004640 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 02:50:46.004754 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16500,"mutex_wait_us":23,"thread_start_us":12502,"threads_started":1}
I20260619 02:50:46.006124 23560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:45.982839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:33559 (local address 127.0.0.1:54470)
0619 02:50:45.998375 (+ 15536us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:45.998396 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:45.998513 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:45.999406 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:45.999410 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:45.999432 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:45.999752 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:45.999760 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:46.001635 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:46.001641 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:46.003612 (+ 1971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:46.003626 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:46.005426 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:46.005433 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:46.005599 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:46.005630 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 02:50:46.005722 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 02:50:46.005835 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15382,"thread_start_us":15252,"threads_started":1}
I20260619 02:50:46.006196 23561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:45.983317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54470 (local address 127.18.134.3:33559)
0619 02:50:45.997919 (+ 14602us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:45.997929 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:45.998657 (+ 728us) server_negotiation.cc:407] Connection header received
0619 02:50:45.998759 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:45.998766 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:45.998987 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:45.999162 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:45.999938 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:46.001493 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:46.003786 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:46.004963 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:46.005081 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:46.005284 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:46.005778 (+ 494us) server_negotiation.cc:299] Negotiation successful
0619 02:50:46.005948 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14427,"thread_start_us":14293,"threads_started":1}
I20260619 02:50:46.006716 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:45.981114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44742 (local address 127.18.134.2:44277)
0619 02:50:45.996930 (+ 15816us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:45.996939 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:45.997770 (+ 831us) server_negotiation.cc:407] Connection header received
0619 02:50:45.997868 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:45.997875 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:45.998126 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:45.999239 (+ 1113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:45.999456 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:46.000686 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:46.002587 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:46.003977 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:46.004104 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:46.004250 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:46.004703 (+ 453us) server_negotiation.cc:299] Negotiation successful
0619 02:50:46.004872 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15661,"thread_start_us":15833,"threads_started":1}
I20260619 02:50:46.008162 23214 tablet_service.cc:1467] Processing CreateTablet for tablet f808754d812a4983a8861b913af4f8c4 (DEFAULT_TABLE table=test-table [id=49813308649244c19973287da9342228]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:50:46.009282 23279 heartbeater.cc:503] Master 127.18.134.62:34835 was elected leader, sending a full tablet report...
I20260619 02:50:46.009450 23214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f808754d812a4983a8861b913af4f8c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:46.009985 23479 tablet_service.cc:1467] Processing CreateTablet for tablet f808754d812a4983a8861b913af4f8c4 (DEFAULT_TABLE table=test-table [id=49813308649244c19973287da9342228]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:50:46.009985 23347 tablet_service.cc:1467] Processing CreateTablet for tablet f808754d812a4983a8861b913af4f8c4 (DEFAULT_TABLE table=test-table [id=49813308649244c19973287da9342228]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:50:46.011336 23347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f808754d812a4983a8861b913af4f8c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:46.011333 23479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f808754d812a4983a8861b913af4f8c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:46.020954 23562 tablet_bootstrap.cc:492] T f808754d812a4983a8861b913af4f8c4 P 73d263e596b94d8991c79adccd0e0f1e: Bootstrap starting.
I20260619 02:50:46.024039 23562 tablet_bootstrap.cc:654] T f808754d812a4983a8861b913af4f8c4 P 73d263e596b94d8991c79adccd0e0f1e: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:46.025202 23563 tablet_bootstrap.cc:492] T f808754d812a4983a8861b913af4f8c4 P 00a18ff270ea4dc99518c40fda65f190: Bootstrap starting.
I20260619 02:50:46.025974 23564 tablet_bootstrap.cc:492] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b: Bootstrap starting.
I20260619 02:50:46.026528 23562 log.cc:824] T f808754d812a4983a8861b913af4f8c4 P 73d263e596b94d8991c79adccd0e0f1e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:46.029299 23562 tablet_bootstrap.cc:492] T f808754d812a4983a8861b913af4f8c4 P 73d263e596b94d8991c79adccd0e0f1e: No bootstrap required, opened a new log
I20260619 02:50:46.029551 23562 ts_tablet_manager.cc:1397] T f808754d812a4983a8861b913af4f8c4 P 73d263e596b94d8991c79adccd0e0f1e: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260619 02:50:46.029553 23564 tablet_bootstrap.cc:654] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:46.030488 23563 tablet_bootstrap.cc:654] T f808754d812a4983a8861b913af4f8c4 P 00a18ff270ea4dc99518c40fda65f190: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:46.030772 23564 log.cc:824] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:46.031607 23563 log.cc:824] T f808754d812a4983a8861b913af4f8c4 P 00a18ff270ea4dc99518c40fda65f190: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:46.033864 23564 tablet_bootstrap.cc:492] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b: No bootstrap required, opened a new log
I20260619 02:50:46.034215 23564 ts_tablet_manager.cc:1397] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260619 02:50:46.037279 23563 tablet_bootstrap.cc:492] T f808754d812a4983a8861b913af4f8c4 P 00a18ff270ea4dc99518c40fda65f190: No bootstrap required, opened a new log
I20260619 02:50:46.037231 23562 raft_consensus.cc:348] T f808754d812a4983a8861b913af4f8c4 P 73d263e596b94d8991c79adccd0e0f1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73d263e596b94d8991c79adccd0e0f1e" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 44077 } } peers { permanent_uuid: "00a18ff270ea4dc99518c40fda65f190" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 44277 } } peers { permanent_uuid: "e6272816ff2c4b29b86ae64c1f6da25b" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33559 } }
I20260619 02:50:46.037541 23563 ts_tablet_manager.cc:1397] T f808754d812a4983a8861b913af4f8c4 P 00a18ff270ea4dc99518c40fda65f190: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.000s
I20260619 02:50:46.037595 23562 raft_consensus.cc:374] T f808754d812a4983a8861b913af4f8c4 P 73d263e596b94d8991c79adccd0e0f1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:46.037703 23562 raft_consensus.cc:729] T f808754d812a4983a8861b913af4f8c4 P 73d263e596b94d8991c79adccd0e0f1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73d263e596b94d8991c79adccd0e0f1e, State: Initialized, Role: FOLLOWER
I20260619 02:50:46.038260 23562 consensus_queue.cc:260] T f808754d812a4983a8861b913af4f8c4 P 73d263e596b94d8991c79adccd0e0f1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73d263e596b94d8991c79adccd0e0f1e" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 44077 } } peers { permanent_uuid: "00a18ff270ea4dc99518c40fda65f190" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 44277 } } peers { permanent_uuid: "e6272816ff2c4b29b86ae64c1f6da25b" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33559 } }
I20260619 02:50:46.039721 23562 ts_tablet_manager.cc:1428] T f808754d812a4983a8861b913af4f8c4 P 73d263e596b94d8991c79adccd0e0f1e: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260619 02:50:46.042712 23564 raft_consensus.cc:348] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73d263e596b94d8991c79adccd0e0f1e" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 44077 } } peers { permanent_uuid: "00a18ff270ea4dc99518c40fda65f190" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 44277 } } peers { permanent_uuid: "e6272816ff2c4b29b86ae64c1f6da25b" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33559 } }
I20260619 02:50:46.043072 23564 raft_consensus.cc:374] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:46.043179 23564 raft_consensus.cc:729] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6272816ff2c4b29b86ae64c1f6da25b, State: Initialized, Role: FOLLOWER
I20260619 02:50:46.043818 23564 consensus_queue.cc:260] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73d263e596b94d8991c79adccd0e0f1e" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 44077 } } peers { permanent_uuid: "00a18ff270ea4dc99518c40fda65f190" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 44277 } } peers { permanent_uuid: "e6272816ff2c4b29b86ae64c1f6da25b" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33559 } }
I20260619 02:50:46.044925 23544 heartbeater.cc:503] Master 127.18.134.62:34835 was elected leader, sending a full tablet report...
I20260619 02:50:46.045644 23564 ts_tablet_manager.cc:1428] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260619 02:50:46.047921 23563 raft_consensus.cc:348] T f808754d812a4983a8861b913af4f8c4 P 00a18ff270ea4dc99518c40fda65f190 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73d263e596b94d8991c79adccd0e0f1e" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 44077 } } peers { permanent_uuid: "00a18ff270ea4dc99518c40fda65f190" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 44277 } } peers { permanent_uuid: "e6272816ff2c4b29b86ae64c1f6da25b" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33559 } }
I20260619 02:50:46.048237 23563 raft_consensus.cc:374] T f808754d812a4983a8861b913af4f8c4 P 00a18ff270ea4dc99518c40fda65f190 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:46.048328 23563 raft_consensus.cc:729] T f808754d812a4983a8861b913af4f8c4 P 00a18ff270ea4dc99518c40fda65f190 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00a18ff270ea4dc99518c40fda65f190, State: Initialized, Role: FOLLOWER
I20260619 02:50:46.048887 23563 consensus_queue.cc:260] T f808754d812a4983a8861b913af4f8c4 P 00a18ff270ea4dc99518c40fda65f190 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73d263e596b94d8991c79adccd0e0f1e" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 44077 } } peers { permanent_uuid: "00a18ff270ea4dc99518c40fda65f190" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 44277 } } peers { permanent_uuid: "e6272816ff2c4b29b86ae64c1f6da25b" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33559 } }
I20260619 02:50:46.049813 23412 heartbeater.cc:503] Master 127.18.134.62:34835 was elected leader, sending a full tablet report...
I20260619 02:50:46.050230 23563 ts_tablet_manager.cc:1428] T f808754d812a4983a8861b913af4f8c4 P 00a18ff270ea4dc99518c40fda65f190: Time spent starting tablet: real 0.013s user 0.006s sys 0.004s
W20260619 02:50:46.158383 23545 tablet.cc:2369] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:50:46.199661 23413 tablet.cc:2369] T f808754d812a4983a8861b913af4f8c4 P 00a18ff270ea4dc99518c40fda65f190: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:50:46.223583 23569 raft_consensus.cc:482] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:50:46.223872 23569 raft_consensus.cc:504] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73d263e596b94d8991c79adccd0e0f1e" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 44077 } } peers { permanent_uuid: "00a18ff270ea4dc99518c40fda65f190" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 44277 } } peers { permanent_uuid: "e6272816ff2c4b29b86ae64c1f6da25b" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33559 } }
I20260619 02:50:46.225592 23569 leader_election.cc:288] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73d263e596b94d8991c79adccd0e0f1e (127.18.134.1:44077), 00a18ff270ea4dc99518c40fda65f190 (127.18.134.2:44277)
W20260619 02:50:46.231552 23280 tablet.cc:2369] T f808754d812a4983a8861b913af4f8c4 P 73d263e596b94d8991c79adccd0e0f1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:50:46.232118 23546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:46.226070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:44077 (local address 127.18.134.3:49873)
0619 02:50:46.226291 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:46.226311 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:46.226429 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:46.226765 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:46.226774 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:46.226802 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:46.227072 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:46.227093 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:46.228380 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:46.228386 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:46.229968 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:46.229979 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:46.231449 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:46.231456 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:46.231634 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:46.231656 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 02:50:46.231741 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 02:50:46.231858 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260619 02:50:46.232180 23558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:46.226070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:49873 (local address 127.18.134.1:44077)
0619 02:50:46.226334 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:46.226340 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:46.226359 (+ 19us) server_negotiation.cc:407] Connection header received
0619 02:50:46.226551 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:46.226554 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:46.226613 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:46.226725 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:46.227257 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:46.228245 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:46.230138 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:46.231331 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:46.231477 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:46.231552 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:46.231850 (+ 298us) server_negotiation.cc:299] Negotiation successful
0619 02:50:46.231965 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260619 02:50:46.233093 23571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:46.226601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:44277 (local address 127.18.134.3:38873)
0619 02:50:46.227249 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:46.227266 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:46.227389 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:46.227645 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:46.227649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:46.227673 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:50:46.227966 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:46.227975 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:46.229311 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:46.229314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:46.231069 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:46.231082 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:46.232474 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:46.232481 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:46.232646 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:46.232678 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 02:50:46.232764 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 02:50:46.232850 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"mutex_wait_us":3,"thread_start_us":357,"threads_started":1}
I20260619 02:50:46.233141 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:46.226211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:38873 (local address 127.18.134.2:44277)
0619 02:50:46.226388 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:46.226393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:46.227349 (+ 956us) server_negotiation.cc:407] Connection header received
0619 02:50:46.227489 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:46.227493 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:46.227547 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:46.227614 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:50:46.228182 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:46.229165 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:46.231200 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:46.232273 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:46.232407 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:46.232470 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:46.232800 (+ 330us) server_negotiation.cc:299] Negotiation successful
0619 02:50:46.232919 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 02:50:46.233274 23234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f808754d812a4983a8861b913af4f8c4" candidate_uuid: "e6272816ff2c4b29b86ae64c1f6da25b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73d263e596b94d8991c79adccd0e0f1e" is_pre_election: true
I20260619 02:50:46.233819 23234 raft_consensus.cc:2454] T f808754d812a4983a8861b913af4f8c4 P 73d263e596b94d8991c79adccd0e0f1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6272816ff2c4b29b86ae64c1f6da25b in term 0.
I20260619 02:50:46.234117 23367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f808754d812a4983a8861b913af4f8c4" candidate_uuid: "e6272816ff2c4b29b86ae64c1f6da25b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00a18ff270ea4dc99518c40fda65f190" is_pre_election: true
I20260619 02:50:46.234521 23434 leader_election.cc:302] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73d263e596b94d8991c79adccd0e0f1e, e6272816ff2c4b29b86ae64c1f6da25b; no voters:
I20260619 02:50:46.234680 23367 raft_consensus.cc:2454] T f808754d812a4983a8861b913af4f8c4 P 00a18ff270ea4dc99518c40fda65f190 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6272816ff2c4b29b86ae64c1f6da25b in term 0.
I20260619 02:50:46.234967 23569 raft_consensus.cc:2781] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:50:46.235103 23569 raft_consensus.cc:482] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:50:46.235208 23569 raft_consensus.cc:3037] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:46.237190 23569 raft_consensus.cc:504] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73d263e596b94d8991c79adccd0e0f1e" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 44077 } } peers { permanent_uuid: "00a18ff270ea4dc99518c40fda65f190" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 44277 } } peers { permanent_uuid: "e6272816ff2c4b29b86ae64c1f6da25b" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33559 } }
I20260619 02:50:46.237725 23569 leader_election.cc:288] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b [CANDIDATE]: Term 1 election: Requested vote from peers 73d263e596b94d8991c79adccd0e0f1e (127.18.134.1:44077), 00a18ff270ea4dc99518c40fda65f190 (127.18.134.2:44277)
I20260619 02:50:46.238157 23234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f808754d812a4983a8861b913af4f8c4" candidate_uuid: "e6272816ff2c4b29b86ae64c1f6da25b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73d263e596b94d8991c79adccd0e0f1e"
I20260619 02:50:46.238199 23367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f808754d812a4983a8861b913af4f8c4" candidate_uuid: "e6272816ff2c4b29b86ae64c1f6da25b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00a18ff270ea4dc99518c40fda65f190"
I20260619 02:50:46.238339 23234 raft_consensus.cc:3037] T f808754d812a4983a8861b913af4f8c4 P 73d263e596b94d8991c79adccd0e0f1e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:46.238356 23367 raft_consensus.cc:3037] T f808754d812a4983a8861b913af4f8c4 P 00a18ff270ea4dc99518c40fda65f190 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:46.240443 23367 raft_consensus.cc:2454] T f808754d812a4983a8861b913af4f8c4 P 00a18ff270ea4dc99518c40fda65f190 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6272816ff2c4b29b86ae64c1f6da25b in term 1.
I20260619 02:50:46.240446 23234 raft_consensus.cc:2454] T f808754d812a4983a8861b913af4f8c4 P 73d263e596b94d8991c79adccd0e0f1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6272816ff2c4b29b86ae64c1f6da25b in term 1.
I20260619 02:50:46.240816 23432 leader_election.cc:302] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00a18ff270ea4dc99518c40fda65f190, e6272816ff2c4b29b86ae64c1f6da25b; no voters:
I20260619 02:50:46.241091 23569 raft_consensus.cc:2781] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:50:46.241509 23569 raft_consensus.cc:686] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b [term 1 LEADER]: Becoming Leader. State: Replica: e6272816ff2c4b29b86ae64c1f6da25b, State: Running, Role: LEADER
I20260619 02:50:46.241951 23569 consensus_queue.cc:237] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73d263e596b94d8991c79adccd0e0f1e" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 44077 } } peers { permanent_uuid: "00a18ff270ea4dc99518c40fda65f190" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 44277 } } peers { permanent_uuid: "e6272816ff2c4b29b86ae64c1f6da25b" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33559 } }
I20260619 02:50:46.246585 23091 catalog_manager.cc:5495] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b reported cstate change: term changed from 0 to 1, leader changed from <none> to e6272816ff2c4b29b86ae64c1f6da25b (127.18.134.3). New cstate: current_term: 1 leader_uuid: "e6272816ff2c4b29b86ae64c1f6da25b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73d263e596b94d8991c79adccd0e0f1e" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 44077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00a18ff270ea4dc99518c40fda65f190" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 44277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6272816ff2c4b29b86ae64c1f6da25b" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33559 } health_report { overall_health: HEALTHY } } }
I20260619 02:50:46.299384 23561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:46.294535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54474 (local address 127.18.134.3:33559)
0619 02:50:46.294713 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:46.294717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:46.294766 (+ 49us) server_negotiation.cc:407] Connection header received
0619 02:50:46.294997 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:46.295001 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:46.295102 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:46.295226 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:50:46.295570 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:46.296291 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:46.297574 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:46.297936 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:46.298018 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:46.298412 (+ 394us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:50:46.298902 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:50:46.299014 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:46.299136 (+ 122us) server_negotiation.cc:299] Negotiation successful
0619 02:50:46.299199 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 02:50:46.307428 23234 raft_consensus.cc:1261] T f808754d812a4983a8861b913af4f8c4 P 73d263e596b94d8991c79adccd0e0f1e [term 1 FOLLOWER]: Refusing update from remote peer e6272816ff2c4b29b86ae64c1f6da25b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:50:46.307428 23367 raft_consensus.cc:1261] T f808754d812a4983a8861b913af4f8c4 P 00a18ff270ea4dc99518c40fda65f190 [term 1 FOLLOWER]: Refusing update from remote peer e6272816ff2c4b29b86ae64c1f6da25b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:50:46.308374 23572 consensus_queue.cc:1035] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b [LEADER]: Connected to new peer: Peer: permanent_uuid: "00a18ff270ea4dc99518c40fda65f190" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 44277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:50:46.308682 23569 consensus_queue.cc:1035] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b [LEADER]: Connected to new peer: Peer: permanent_uuid: "73d263e596b94d8991c79adccd0e0f1e" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 44077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:50:46.320441 23577 mvcc.cc:205] Tried to move back new op lower bound from 7298406180064407552 to 7298406179810693120. Current Snapshot: MvccSnapshot[applied={T|T < 7298406180064407552}]
I20260619 02:50:46.331686 23091 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:50:46.331913 23091 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:50:46.335100 23091 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 670600acaf6f480dbbe3cb8e55874b42: Sending DeleteTablet for 3 replicas of tablet f808754d812a4983a8861b913af4f8c4
I20260619 02:50:46.336117 23214 tablet_service.cc:1514] Processing DeleteTablet for tablet f808754d812a4983a8861b913af4f8c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:50:46 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:57284
I20260619 02:50:46.336156 23347 tablet_service.cc:1514] Processing DeleteTablet for tablet f808754d812a4983a8861b913af4f8c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:50:46 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:44742
I20260619 02:50:46.336299 23479 tablet_service.cc:1514] Processing DeleteTablet for tablet f808754d812a4983a8861b913af4f8c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:50:46 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:54470
I20260619 02:50:46.343742 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 23152
I20260619 02:50:46.363432 23586 tablet_replica.cc:330] T f808754d812a4983a8861b913af4f8c4 P 00a18ff270ea4dc99518c40fda65f190: stopping tablet replica
I20260619 02:50:46.363948 23586 raft_consensus.cc:2229] T f808754d812a4983a8861b913af4f8c4 P 00a18ff270ea4dc99518c40fda65f190 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:50:46.364503 23586 raft_consensus.cc:2258] T f808754d812a4983a8861b913af4f8c4 P 00a18ff270ea4dc99518c40fda65f190 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:50:46.370041 23586 ts_tablet_manager.cc:1905] T f808754d812a4983a8861b913af4f8c4 P 00a18ff270ea4dc99518c40fda65f190: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 19 02:50:46 dist-test-slave-9mr0 krb5kdc[23047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.1@KRBTEST.COM
I20260619 02:50:46.377714 23587 tablet_replica.cc:330] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b: stopping tablet replica
I20260619 02:50:46.377864 23585 tablet_replica.cc:330] T f808754d812a4983a8861b913af4f8c4 P 73d263e596b94d8991c79adccd0e0f1e: stopping tablet replica
I20260619 02:50:46.378191 23587 raft_consensus.cc:2229] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:50:46.378280 23585 raft_consensus.cc:2229] T f808754d812a4983a8861b913af4f8c4 P 73d263e596b94d8991c79adccd0e0f1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:50:46.378840 23585 raft_consensus.cc:2258] T f808754d812a4983a8861b913af4f8c4 P 73d263e596b94d8991c79adccd0e0f1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:50:46.379016 23587 raft_consensus.cc:2258] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:50:46.380990 23558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:46.344651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57290 (local address 127.18.134.1:44077)
0619 02:50:46.344843 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:46.344850 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:46.359184 (+ 14334us) server_negotiation.cc:407] Connection header received
0619 02:50:46.359459 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:46.359465 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:46.359549 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:46.359642 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:46.361493 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:46.362607 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:46.366429 (+ 3822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:46.367203 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:46.367343 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:46.373020 (+ 5677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:46.373041 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:46.373062 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:46.373118 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:46.377165 (+ 4047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:46.378208 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:46.378218 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:46.378221 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:46.378517 (+ 296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:46.379051 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:46.379056 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:46.379059 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:46.379539 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:46.379845 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:46.380403 (+ 558us) server_negotiation.cc:299] Negotiation successful
0619 02:50:46.380638 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 02:50:46.381040 23587 ts_tablet_manager.cc:1905] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:50:46.381033 23585 ts_tablet_manager.cc:1905] T f808754d812a4983a8861b913af4f8c4 P 73d263e596b94d8991c79adccd0e0f1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:50:46.383540 23586 ts_tablet_manager.cc:1918] T f808754d812a4983a8861b913af4f8c4 P 00a18ff270ea4dc99518c40fda65f190: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:50:46.383709 23586 log.cc:1196] T f808754d812a4983a8861b913af4f8c4 P 00a18ff270ea4dc99518c40fda65f190: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-1/wal/wals/f808754d812a4983a8861b913af4f8c4
I20260619 02:50:46.384244 23586 ts_tablet_manager.cc:1939] T f808754d812a4983a8861b913af4f8c4 P 00a18ff270ea4dc99518c40fda65f190: Deleting consensus metadata
I20260619 02:50:46.385027 23274 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:46.386003 23075 catalog_manager.cc:4841] TS 00a18ff270ea4dc99518c40fda65f190 (127.18.134.2:44277): tablet f808754d812a4983a8861b913af4f8c4 (table test-table [id=49813308649244c19973287da9342228]) successfully deleted
I20260619 02:50:46.388145 23587 ts_tablet_manager.cc:1918] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:50:46.388317 23587 log.cc:1196] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781837410515220-18968-0/minicluster-data/ts-2/wal/wals/f808754d812a4983a8861b913af4f8c4
I20260619 02:50:46.388787 23587 ts_tablet_manager.cc:1939] T f808754d812a4983a8861b913af4f8c4 P e6272816ff2c4b29b86ae64c1f6da25b: Deleting consensus metadata
I20260619 02:50:46.390548 23076 catalog_manager.cc:4841] TS e6272816ff2c4b29b86ae64c1f6da25b (127.18.134.3:33559): tablet f808754d812a4983a8861b913af4f8c4 (table test-table [id=49813308649244c19973287da9342228]) successfully deleted
I20260619 02:50:46.505654 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 23152
W20260619 02:50:46.524726 23077 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.134.1:44077 (error 108)
I20260619 02:50:46.525929 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 23285
I20260619 02:50:46.526787 23591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:46.525884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:44077 (local address 127.0.0.1:57294)
0619 02:50:46.526430 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:46.526567 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:44077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":310,"threads_started":1}
W20260619 02:50:46.527115 23077 catalog_manager.cc:4584] TS 73d263e596b94d8991c79adccd0e0f1e (127.18.134.1:44077): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f808754d812a4983a8861b913af4f8c4: Network error: Client connection negotiation failed: client connection to 127.18.134.1:44077: connect: Connection refused (error 111)
Jun 19 02:50:46 dist-test-slave-9mr0 krb5kdc[23047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.2@KRBTEST.COM
I20260619 02:50:46.541944 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:46.526677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44758 (local address 127.18.134.2:44277)
0619 02:50:46.526833 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:46.526839 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:46.527004 (+ 165us) server_negotiation.cc:407] Connection header received
0619 02:50:46.527083 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:46.527089 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:46.527169 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:46.527286 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:46.528454 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:46.529560 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:46.530977 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:46.531596 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:46.531722 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:46.535792 (+ 4070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:46.535809 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:46.535826 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:46.535866 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:46.539361 (+ 3495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:46.539999 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:46.540009 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:46.540012 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:46.540160 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:46.540487 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:46.540492 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:46.540495 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:46.540908 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:46.541182 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:46.541386 (+ 204us) server_negotiation.cc:299] Negotiation successful
0619 02:50:46.541593 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 02:50:46.542583 23407 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:46.580845 23591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:46.580388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:44077 (local address 127.0.0.1:57310)
0619 02:50:46.580603 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:46.580703 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:44077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260619 02:50:46.582010 23591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:46.581597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:44077 (local address 127.0.0.1:57316)
0619 02:50:46.581745 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:46.581866 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:44077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260619 02:50:46.642414 23591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:46.641973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:44077 (local address 127.0.0.1:57322)
0619 02:50:46.642174 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:46.642254 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:44077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260619 02:50:46.643479 23591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:46.643156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:44077 (local address 127.0.0.1:57334)
0619 02:50:46.643292 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:46.643356 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:44077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 02:50:46.689966 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 23285
I20260619 02:50:46.704211 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 23417
Jun 19 02:50:46 dist-test-slave-9mr0 krb5kdc[23047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.3@KRBTEST.COM
I20260619 02:50:46.722325 23561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:46.705045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54490 (local address 127.18.134.3:33559)
0619 02:50:46.705190 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:46.705194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:46.705246 (+ 52us) server_negotiation.cc:407] Connection header received
0619 02:50:46.705465 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:46.705469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:46.705544 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:46.705627 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:46.707227 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:46.708108 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:46.709866 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:46.710346 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:46.710455 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:46.715804 (+ 5349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:46.715821 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:46.715842 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:46.715900 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:46.719310 (+ 3410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:46.720158 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:46.720166 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:46.720168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:46.720328 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:46.720819 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:46.720823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:46.720825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:46.721206 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:46.721474 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:46.721753 (+ 279us) server_negotiation.cc:299] Negotiation successful
0619 02:50:46.721913 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 02:50:46.723109 23539 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:46.724093 23591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:46.723628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:44077 (local address 127.0.0.1:57346)
0619 02:50:46.723805 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:46.723892 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:44077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260619 02:50:46.725045 23591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:46.724693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:44077 (local address 127.0.0.1:57352)
0619 02:50:46.724851 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:46.724922 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:44077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260619 02:50:46.843000 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 23417
I20260619 02:50:46.858647 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 23060
I20260619 02:50:46.859756 23121 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:46.963902 23591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:46.963461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:44077 (local address 127.0.0.1:57356)
0619 02:50:46.963679 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:46.963764 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:44077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260619 02:50:46.964123 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 23060
2026-06-19T02:50:46Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4137 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-19T02:50:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:50:47Z Disabled control of system clock
I20260619 02:50:47.022786 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:38565
--webserver_interface=127.18.134.62
--webserver_port=0
--builtin_ntp_servers=127.18.134.20:37641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:38565
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 02:50:47.332727 23598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:47.333030 23598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:47.333113 23598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:47.340265 23598 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:50:47.340376 23598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:47.340412 23598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:47.340462 23598 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:50:47.340507 23598 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:50:47.349550 23598 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:37641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:38565
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:38565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:47.350805 23598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:47.352450 23598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:47.363866 23603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:47.363899 23604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:47.363929 23606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:47.364665 23598 server_base.cc:1034] running on GCE node
I20260619 02:50:47.365415 23598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:47.366925 23598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:47.368141 23598 hybrid_clock.cc:648] HybridClock initialized: now 1781837447368081 us; error 56 us; skew 500 ppm
I20260619 02:50:47.368489 23598 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:50:47.371479 23598 webserver.cc:460] Webserver started at http://127.18.134.62:32769/ using document root <none> and password file <none>
I20260619 02:50:47.372165 23598 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:47.372269 23598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:47.372565 23598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:47.375423 23598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "58a32f13e7034cbb8cf2242ad6a81e2d"
format_stamp: "Formatted at 2026-06-19 02:50:47 on dist-test-slave-9mr0"
server_key: "436a9c41668f9f52a66de6700b5b5a95"
server_key_iv: "39fd6e2960d6b6bab57ac9869b792ad2"
server_key_version: "encryptionkey@0"
I20260619 02:50:47.376183 23598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "58a32f13e7034cbb8cf2242ad6a81e2d"
format_stamp: "Formatted at 2026-06-19 02:50:47 on dist-test-slave-9mr0"
server_key: "436a9c41668f9f52a66de6700b5b5a95"
server_key_iv: "39fd6e2960d6b6bab57ac9869b792ad2"
server_key_version: "encryptionkey@0"
I20260619 02:50:47.381289 23598 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260619 02:50:47.384685 23612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:47.386023 23598 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:50:47.386180 23598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "58a32f13e7034cbb8cf2242ad6a81e2d"
format_stamp: "Formatted at 2026-06-19 02:50:47 on dist-test-slave-9mr0"
server_key: "436a9c41668f9f52a66de6700b5b5a95"
server_key_iv: "39fd6e2960d6b6bab57ac9869b792ad2"
server_key_version: "encryptionkey@0"
I20260619 02:50:47.386381 23598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:47.391952 23598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:47.392678 23598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:47.392947 23598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:47.409513 23598 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:38565
I20260619 02:50:47.409547 23663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:38565 every 8 connection(s)
I20260619 02:50:47.410812 23598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:50:47.414484 23664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:47.419548 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 23598
I20260619 02:50:47.419739 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:50:47.420426 18968 external_mini_cluster.cc:1428] Setting key 6940b66b4ca5b5788c47cc5a217170bf
I20260619 02:50:47.425745 23664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58a32f13e7034cbb8cf2242ad6a81e2d: Bootstrap starting.
I20260619 02:50:47.429162 23664 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58a32f13e7034cbb8cf2242ad6a81e2d: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:47.430102 23664 log.cc:824] T 00000000000000000000000000000000 P 58a32f13e7034cbb8cf2242ad6a81e2d: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:47.433107 23664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58a32f13e7034cbb8cf2242ad6a81e2d: No bootstrap required, opened a new log
I20260619 02:50:47.434773 23667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:47.422458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45286 (local address 127.18.134.62:38565)
0619 02:50:47.423632 (+ 1174us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:47.423648 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:47.423696 (+ 48us) server_negotiation.cc:407] Connection header received
0619 02:50:47.425796 (+ 2100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:47.425824 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:47.426335 (+ 511us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:47.426780 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:47.432554 (+ 5774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:47.432564 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:47.432582 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:50:47.432614 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:47.432702 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:50:47.432728 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:47.432953 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:47.433097 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 02:50:47.433402 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":361,"threads_started":1}
I20260619 02:50:47.441217 23664 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58a32f13e7034cbb8cf2242ad6a81e2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a32f13e7034cbb8cf2242ad6a81e2d" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 38565 } }
I20260619 02:50:47.441592 23664 raft_consensus.cc:374] T 00000000000000000000000000000000 P 58a32f13e7034cbb8cf2242ad6a81e2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:47.441679 23664 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58a32f13e7034cbb8cf2242ad6a81e2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58a32f13e7034cbb8cf2242ad6a81e2d, State: Initialized, Role: FOLLOWER
I20260619 02:50:47.442257 23664 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58a32f13e7034cbb8cf2242ad6a81e2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a32f13e7034cbb8cf2242ad6a81e2d" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 38565 } }
I20260619 02:50:47.442480 23664 raft_consensus.cc:388] T 00000000000000000000000000000000 P 58a32f13e7034cbb8cf2242ad6a81e2d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:50:47.442586 23664 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58a32f13e7034cbb8cf2242ad6a81e2d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:50:47.442775 23664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58a32f13e7034cbb8cf2242ad6a81e2d [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:47.445003 23664 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58a32f13e7034cbb8cf2242ad6a81e2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a32f13e7034cbb8cf2242ad6a81e2d" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 38565 } }
I20260619 02:50:47.445483 23664 leader_election.cc:302] T 00000000000000000000000000000000 P 58a32f13e7034cbb8cf2242ad6a81e2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58a32f13e7034cbb8cf2242ad6a81e2d; no voters:
I20260619 02:50:47.446092 23664 leader_election.cc:288] T 00000000000000000000000000000000 P 58a32f13e7034cbb8cf2242ad6a81e2d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:50:47.446182 23669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58a32f13e7034cbb8cf2242ad6a81e2d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:50:47.446892 23669 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58a32f13e7034cbb8cf2242ad6a81e2d [term 1 LEADER]: Becoming Leader. State: Replica: 58a32f13e7034cbb8cf2242ad6a81e2d, State: Running, Role: LEADER
I20260619 02:50:47.447408 23669 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58a32f13e7034cbb8cf2242ad6a81e2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a32f13e7034cbb8cf2242ad6a81e2d" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 38565 } }
I20260619 02:50:47.448204 23664 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58a32f13e7034cbb8cf2242ad6a81e2d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:50:47.449895 23670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58a32f13e7034cbb8cf2242ad6a81e2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58a32f13e7034cbb8cf2242ad6a81e2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a32f13e7034cbb8cf2242ad6a81e2d" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 38565 } } }
I20260619 02:50:47.449872 23671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58a32f13e7034cbb8cf2242ad6a81e2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58a32f13e7034cbb8cf2242ad6a81e2d. Latest consensus state: current_term: 1 leader_uuid: "58a32f13e7034cbb8cf2242ad6a81e2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58a32f13e7034cbb8cf2242ad6a81e2d" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 38565 } } }
I20260619 02:50:47.450271 23670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58a32f13e7034cbb8cf2242ad6a81e2d [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:47.450378 23671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58a32f13e7034cbb8cf2242ad6a81e2d [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:47.451022 23674 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:50:47.456909 23674 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:50:47.466847 23674 catalog_manager.cc:1347] Generated new cluster ID: dff9baa76b944c9ca00bffa1bd22c51f
I20260619 02:50:47.467020 23674 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:50:47.482018 23674 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:50:47.482437 23674 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:50:47.490350 23674 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58a32f13e7034cbb8cf2242ad6a81e2d: Generated new TSK 0
I20260619 02:50:47.491466 23674 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:50:47.506165 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:0
--local_ip_for_outbound_sockets=127.18.134.1
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:38565
--builtin_ntp_servers=127.18.134.20:37641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 02:50:47.816351 23688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:47.816620 23688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:47.816709 23688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:47.823828 23688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:47.823932 23688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:47.824048 23688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:50:47.832939 23688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:37641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:38565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:47.834125 23688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:47.835780 23688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:47.847815 23693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:47.847779 23696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:47.847892 23694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:47.848747 23688 server_base.cc:1034] running on GCE node
I20260619 02:50:47.849370 23688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:47.850312 23688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:47.851491 23688 hybrid_clock.cc:648] HybridClock initialized: now 1781837447851428 us; error 53 us; skew 500 ppm
I20260619 02:50:47.851789 23688 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:50:47.854033 23688 webserver.cc:460] Webserver started at http://127.18.134.1:40259/ using document root <none> and password file <none>
I20260619 02:50:47.854722 23688 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:47.854808 23688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:47.855105 23688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:47.857745 23688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-0/data/instance:
uuid: "232164e4fda94b69beaabf55732722a8"
format_stamp: "Formatted at 2026-06-19 02:50:47 on dist-test-slave-9mr0"
server_key: "1fce2111b8399298c91ea9b1439b0a91"
server_key_iv: "2a99e6f7c47e7569369017ef7a6c3353"
server_key_version: "encryptionkey@0"
I20260619 02:50:47.858417 23688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance:
uuid: "232164e4fda94b69beaabf55732722a8"
format_stamp: "Formatted at 2026-06-19 02:50:47 on dist-test-slave-9mr0"
server_key: "1fce2111b8399298c91ea9b1439b0a91"
server_key_iv: "2a99e6f7c47e7569369017ef7a6c3353"
server_key_version: "encryptionkey@0"
I20260619 02:50:47.863384 23688 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260619 02:50:47.866860 23702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:47.868186 23688 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 02:50:47.868321 23688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "232164e4fda94b69beaabf55732722a8"
format_stamp: "Formatted at 2026-06-19 02:50:47 on dist-test-slave-9mr0"
server_key: "1fce2111b8399298c91ea9b1439b0a91"
server_key_iv: "2a99e6f7c47e7569369017ef7a6c3353"
server_key_version: "encryptionkey@0"
I20260619 02:50:47.868525 23688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:47.873811 23688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:47.874536 23688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:47.874775 23688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:47.875586 23688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:50:47.876837 23688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:47.876919 23688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:47.877018 23688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:47.877090 23688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:47.905620 23688 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:44653
I20260619 02:50:47.905689 23814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:44653 every 8 connection(s)
I20260619 02:50:47.906935 23688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:50:47.912966 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 23688
I20260619 02:50:47.913125 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance
I20260619 02:50:47.913828 18968 external_mini_cluster.cc:1428] Setting key 35e40b3b9213b8b2e334839b69b120bb
I20260619 02:50:47.914858 23667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:47.909374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:35305 (local address 127.18.134.62:38565)
0619 02:50:47.909540 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:47.909545 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:47.910934 (+ 1389us) server_negotiation.cc:407] Connection header received
0619 02:50:47.912560 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:47.912565 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:47.912632 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:47.912719 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:47.914122 (+ 1403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:47.914128 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:47.914134 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:50:47.914165 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:47.914196 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:50:47.914207 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:47.914320 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:47.914507 (+ 187us) server_negotiation.cc:299] Negotiation successful
0619 02:50:47.914573 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 02:50:47.915817 23817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:47.909855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:38565 (local address 127.18.134.1:35305)
0619 02:50:47.910791 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:47.910826 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:47.912289 (+ 1463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:47.912950 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:47.912959 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:47.913061 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:47.913802 (+ 741us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:50:47.913804 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:47.913841 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:50:47.913947 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:47.914290 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:47.914301 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:47.914308 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 02:50:47.914578 (+ 270us) client_negotiation.cc:240] Negotiation successful
0619 02:50:47.914871 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":269,"threads_started":1}
I20260619 02:50:47.916613 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:0
--local_ip_for_outbound_sockets=127.18.134.2
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:38565
--builtin_ntp_servers=127.18.134.20:37641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260619 02:50:47.917241 23815 heartbeater.cc:347] Connected to a master server at 127.18.134.62:38565
I20260619 02:50:47.917515 23815 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:47.918061 23815 heartbeater.cc:511] Master 127.18.134.62:38565 requested a full tablet report, sending...
I20260619 02:50:47.920121 23629 ts_manager.cc:194] Registered new tserver with Master: 232164e4fda94b69beaabf55732722a8 (127.18.134.1:44653)
W20260619 02:50:48.223840 23818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:48.224135 23818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:48.224223 23818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:48.231284 23818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:48.231392 23818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:48.231501 23818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:50:48.240361 23818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:37641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:38565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:48.241573 23818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:48.243286 23818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:48.256090 23823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:48.255992 23826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:48.256023 23824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:48.257141 23818 server_base.cc:1034] running on GCE node
I20260619 02:50:48.257700 23818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:48.258741 23818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:48.259948 23818 hybrid_clock.cc:648] HybridClock initialized: now 1781837448259892 us; error 57 us; skew 500 ppm
I20260619 02:50:48.260375 23818 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:50:48.262791 23818 webserver.cc:460] Webserver started at http://127.18.134.2:43459/ using document root <none> and password file <none>
I20260619 02:50:48.263571 23818 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:48.263659 23818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:48.263968 23818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:48.266794 23818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-1/data/instance:
uuid: "522553fed19e486a92da24d0c7fb1e19"
format_stamp: "Formatted at 2026-06-19 02:50:48 on dist-test-slave-9mr0"
server_key: "a72f4833ae007bcdf6f69492672f470a"
server_key_iv: "7ae6c71c97ae74e975c1ef29e619d66b"
server_key_version: "encryptionkey@0"
I20260619 02:50:48.267573 23818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance:
uuid: "522553fed19e486a92da24d0c7fb1e19"
format_stamp: "Formatted at 2026-06-19 02:50:48 on dist-test-slave-9mr0"
server_key: "a72f4833ae007bcdf6f69492672f470a"
server_key_iv: "7ae6c71c97ae74e975c1ef29e619d66b"
server_key_version: "encryptionkey@0"
I20260619 02:50:48.272717 23818 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:50:48.276196 23832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:48.277649 23818 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 02:50:48.277817 23818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "522553fed19e486a92da24d0c7fb1e19"
format_stamp: "Formatted at 2026-06-19 02:50:48 on dist-test-slave-9mr0"
server_key: "a72f4833ae007bcdf6f69492672f470a"
server_key_iv: "7ae6c71c97ae74e975c1ef29e619d66b"
server_key_version: "encryptionkey@0"
I20260619 02:50:48.278026 23818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:48.283712 23818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:48.284541 23818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:48.284788 23818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:48.285697 23818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:50:48.287155 23818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:48.287240 23818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:48.287339 23818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:48.287412 23818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:48.317090 23818 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:37865
I20260619 02:50:48.317132 23944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:37865 every 8 connection(s)
I20260619 02:50:48.318356 23818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:50:48.324836 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 23818
I20260619 02:50:48.325021 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance
I20260619 02:50:48.325554 18968 external_mini_cluster.cc:1428] Setting key 8d056219842a51e7dcdcbeb84d056d20
I20260619 02:50:48.326753 23667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:48.320820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:36999 (local address 127.18.134.62:38565)
0619 02:50:48.320982 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:48.320987 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:48.322556 (+ 1569us) server_negotiation.cc:407] Connection header received
0619 02:50:48.324302 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:48.324307 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:48.324378 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:48.324461 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:48.326032 (+ 1571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:48.326038 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:48.326041 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:50:48.326060 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:48.326092 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:50:48.326104 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:48.326233 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:48.326407 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 02:50:48.326458 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 02:50:48.327942 23947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:48.321368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:38565 (local address 127.18.134.2:36999)
0619 02:50:48.322397 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:48.322434 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:48.324049 (+ 1615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:48.324741 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:48.324753 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:48.324948 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:48.325656 (+ 708us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:50:48.325660 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:48.325713 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:50:48.325840 (+ 127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:48.326206 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:48.326214 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:48.326220 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 02:50:48.326492 (+ 272us) client_negotiation.cc:240] Negotiation successful
0619 02:50:48.326775 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":329,"threads_started":1}
I20260619 02:50:48.328641 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:0
--local_ip_for_outbound_sockets=127.18.134.3
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:38565
--builtin_ntp_servers=127.18.134.20:37641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260619 02:50:48.329378 23945 heartbeater.cc:347] Connected to a master server at 127.18.134.62:38565
I20260619 02:50:48.329677 23945 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:48.330248 23945 heartbeater.cc:511] Master 127.18.134.62:38565 requested a full tablet report, sending...
I20260619 02:50:48.331621 23629 ts_manager.cc:194] Registered new tserver with Master: 522553fed19e486a92da24d0c7fb1e19 (127.18.134.2:37865)
W20260619 02:50:48.639331 23948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:48.639628 23948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:48.639755 23948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:48.647233 23948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:48.647336 23948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:48.647471 23948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:50:48.656816 23948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:37641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:38565
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:48.658033 23948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:48.659791 23948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:48.671989 23953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:48.671901 23956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:48.671926 23954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:48.672477 23948 server_base.cc:1034] running on GCE node
I20260619 02:50:48.673151 23948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:48.674018 23948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:48.675215 23948 hybrid_clock.cc:648] HybridClock initialized: now 1781837448675163 us; error 44 us; skew 500 ppm
I20260619 02:50:48.675514 23948 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:50:48.677728 23948 webserver.cc:460] Webserver started at http://127.18.134.3:44579/ using document root <none> and password file <none>
I20260619 02:50:48.678381 23948 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:48.678474 23948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:48.678817 23948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:48.681605 23948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-2/data/instance:
uuid: "1dda19a58ad4407ea8b3132be6584367"
format_stamp: "Formatted at 2026-06-19 02:50:48 on dist-test-slave-9mr0"
server_key: "8f6542c0c9ba4304884047e610ac9045"
server_key_iv: "3d97598e25d523ae852060d1d6832ae9"
server_key_version: "encryptionkey@0"
I20260619 02:50:48.682399 23948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance:
uuid: "1dda19a58ad4407ea8b3132be6584367"
format_stamp: "Formatted at 2026-06-19 02:50:48 on dist-test-slave-9mr0"
server_key: "8f6542c0c9ba4304884047e610ac9045"
server_key_iv: "3d97598e25d523ae852060d1d6832ae9"
server_key_version: "encryptionkey@0"
I20260619 02:50:48.687342 23948 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 02:50:48.690893 23962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:48.692260 23948 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 02:50:48.692421 23948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "1dda19a58ad4407ea8b3132be6584367"
format_stamp: "Formatted at 2026-06-19 02:50:48 on dist-test-slave-9mr0"
server_key: "8f6542c0c9ba4304884047e610ac9045"
server_key_iv: "3d97598e25d523ae852060d1d6832ae9"
server_key_version: "encryptionkey@0"
I20260619 02:50:48.692636 23948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:48.698065 23948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:48.698829 23948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:48.699057 23948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:48.699940 23948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:50:48.701282 23948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:48.701375 23948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:48.701483 23948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:48.701548 23948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:48.731153 23948 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:41497
I20260619 02:50:48.731206 24074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:41497 every 8 connection(s)
I20260619 02:50:48.732390 23948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:50:48.735630 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 23948
I20260619 02:50:48.735796 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance
I20260619 02:50:48.736320 18968 external_mini_cluster.cc:1428] Setting key a54f68eae390692ea26a6dcc3a86ba6f
I20260619 02:50:48.740684 23667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:48.734864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:54921 (local address 127.18.134.62:38565)
0619 02:50:48.735028 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:48.735032 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:48.736640 (+ 1608us) server_negotiation.cc:407] Connection header received
0619 02:50:48.738670 (+ 2030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:48.738675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:48.738738 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:48.738841 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:48.740175 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:48.740179 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:48.740182 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:50:48.740196 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:48.740224 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:50:48.740234 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:48.740316 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:48.740450 (+ 134us) server_negotiation.cc:299] Negotiation successful
0619 02:50:48.740494 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 02:50:48.741573 24077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:48.735363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:38565 (local address 127.18.134.3:54921)
0619 02:50:48.736459 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:48.736517 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:48.738403 (+ 1886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:48.739135 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:48.739147 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:48.739275 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:48.739913 (+ 638us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:50:48.739915 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:48.739952 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:50:48.740037 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:48.740303 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:48.740308 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:48.740313 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:50:48.740505 (+ 192us) client_negotiation.cc:240] Negotiation successful
0619 02:50:48.740694 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":284,"threads_started":1}
I20260619 02:50:48.742676 24075 heartbeater.cc:347] Connected to a master server at 127.18.134.62:38565
I20260619 02:50:48.742939 24075 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:48.743458 24075 heartbeater.cc:511] Master 127.18.134.62:38565 requested a full tablet report, sending...
I20260619 02:50:48.744632 23629 ts_manager.cc:194] Registered new tserver with Master: 1dda19a58ad4407ea8b3132be6584367 (127.18.134.3:41497)
I20260619 02:50:48.751107 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:50:48.757696 23667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:48.755321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45300 (local address 127.18.134.62:38565)
0619 02:50:48.755494 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:48.755498 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:48.756019 (+ 521us) server_negotiation.cc:407] Connection header received
0619 02:50:48.756130 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:48.756134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:48.756187 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:48.756273 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:48.757144 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:48.757149 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:48.757152 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:50:48.757178 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:48.757209 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:50:48.757220 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:48.757321 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:48.757380 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 02:50:48.757423 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 02:50:48.761505 23629 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45300:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:50:48.768052 23629 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:50:48.786451 24089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:48.780108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60900 (local address 127.18.134.3:41497)
0619 02:50:48.783114 (+ 3006us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:48.783121 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:48.783157 (+ 36us) server_negotiation.cc:407] Connection header received
0619 02:50:48.783228 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:48.783236 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:48.783489 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:48.784333 (+ 844us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:48.785629 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:48.785637 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:48.785652 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:50:48.785667 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:48.785722 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:50:48.785741 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:48.785922 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:48.785989 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 02:50:48.786084 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2854,"thread_start_us":2697,"threads_started":1}
I20260619 02:50:48.786607 24091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:48.780736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39642 (local address 127.18.134.1:44653)
0619 02:50:48.781333 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:48.781341 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:48.783533 (+ 2192us) server_negotiation.cc:407] Connection header received
0619 02:50:48.783619 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:48.783627 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:48.783849 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:48.784009 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:48.785795 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:48.785803 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:48.785816 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:50:48.785844 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:48.785898 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:50:48.785917 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:48.786043 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:48.786242 (+ 199us) server_negotiation.cc:299] Negotiation successful
0619 02:50:48.786330 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":611,"threads_started":1}
I20260619 02:50:48.786607 24087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:48.779733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:41497 (local address 127.0.0.1:60900)
0619 02:50:48.780430 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:48.780469 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:48.780602 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:48.783698 (+ 3096us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:48.783706 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:48.783822 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:48.784646 (+ 824us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:50:48.784649 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:48.784678 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:50:48.784761 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:48.785839 (+ 1078us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:48.785848 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:48.785855 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 02:50:48.786009 (+ 154us) client_negotiation.cc:240] Negotiation successful
0619 02:50:48.786097 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":1901,"threads_started":1}
I20260619 02:50:48.787456 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:48.781179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36946 (local address 127.18.134.2:37865)
0619 02:50:48.781672 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:48.781679 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:48.784049 (+ 2370us) server_negotiation.cc:407] Connection header received
0619 02:50:48.784276 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:48.784284 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:48.784498 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:48.784615 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:48.785969 (+ 1354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:48.785976 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:48.785987 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:50:48.786012 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:48.786064 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:50:48.786082 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:48.786203 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:48.787029 (+ 826us) server_negotiation.cc:299] Negotiation successful
0619 02:50:48.787117 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":325,"threads_started":1}
I20260619 02:50:48.787540 24088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:48.780356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:37865 (local address 127.0.0.1:36946)
0619 02:50:48.783937 (+ 3581us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:48.783955 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:48.784132 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:48.785158 (+ 1026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:48.785162 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:48.785183 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:48.785761 (+ 578us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:50:48.785764 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:48.785778 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:50:48.785816 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:48.786289 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:48.786293 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:48.786297 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:50:48.786444 (+ 147us) client_negotiation.cc:240] Negotiation successful
0619 02:50:48.786493 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3468,"thread_start_us":2770,"threads_started":1}
I20260619 02:50:48.788112 24090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:48.779822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:44653 (local address 127.0.0.1:39642)
0619 02:50:48.783400 (+ 3578us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:48.783419 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:48.783518 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:48.784716 (+ 1198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:48.784720 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:48.784742 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:48.785559 (+ 817us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:50:48.785562 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:48.785577 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:50:48.785618 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:48.786089 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:48.786093 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:48.786097 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:50:48.787021 (+ 924us) client_negotiation.cc:240] Negotiation successful
0619 02:50:48.787067 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3446,"thread_start_us":3868,"threads_started":1}
I20260619 02:50:48.790601 24010 tablet_service.cc:1467] Processing CreateTablet for tablet a14a258ed6f74038a34a3629e7276a75 (DEFAULT_TABLE table=test-table [id=874e0452fd054e21b549a5fb4056c7a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:50:48.791239 23750 tablet_service.cc:1467] Processing CreateTablet for tablet a14a258ed6f74038a34a3629e7276a75 (DEFAULT_TABLE table=test-table [id=874e0452fd054e21b549a5fb4056c7a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:50:48.791234 23880 tablet_service.cc:1467] Processing CreateTablet for tablet a14a258ed6f74038a34a3629e7276a75 (DEFAULT_TABLE table=test-table [id=874e0452fd054e21b549a5fb4056c7a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:50:48.792181 24010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a14a258ed6f74038a34a3629e7276a75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:48.792351 23750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a14a258ed6f74038a34a3629e7276a75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:48.792507 23880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a14a258ed6f74038a34a3629e7276a75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:48.803591 24093 tablet_bootstrap.cc:492] T a14a258ed6f74038a34a3629e7276a75 P 232164e4fda94b69beaabf55732722a8: Bootstrap starting.
I20260619 02:50:48.806347 24095 tablet_bootstrap.cc:492] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19: Bootstrap starting.
I20260619 02:50:48.807618 24094 tablet_bootstrap.cc:492] T a14a258ed6f74038a34a3629e7276a75 P 1dda19a58ad4407ea8b3132be6584367: Bootstrap starting.
I20260619 02:50:48.808625 24093 tablet_bootstrap.cc:654] T a14a258ed6f74038a34a3629e7276a75 P 232164e4fda94b69beaabf55732722a8: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:48.809881 24093 log.cc:824] T a14a258ed6f74038a34a3629e7276a75 P 232164e4fda94b69beaabf55732722a8: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:48.811009 24094 tablet_bootstrap.cc:654] T a14a258ed6f74038a34a3629e7276a75 P 1dda19a58ad4407ea8b3132be6584367: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:48.812134 24094 log.cc:824] T a14a258ed6f74038a34a3629e7276a75 P 1dda19a58ad4407ea8b3132be6584367: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:48.812544 24095 tablet_bootstrap.cc:654] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:48.813623 24095 log.cc:824] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:48.813723 24093 tablet_bootstrap.cc:492] T a14a258ed6f74038a34a3629e7276a75 P 232164e4fda94b69beaabf55732722a8: No bootstrap required, opened a new log
I20260619 02:50:48.814139 24093 ts_tablet_manager.cc:1397] T a14a258ed6f74038a34a3629e7276a75 P 232164e4fda94b69beaabf55732722a8: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260619 02:50:48.815106 24094 tablet_bootstrap.cc:492] T a14a258ed6f74038a34a3629e7276a75 P 1dda19a58ad4407ea8b3132be6584367: No bootstrap required, opened a new log
I20260619 02:50:48.815351 24094 ts_tablet_manager.cc:1397] T a14a258ed6f74038a34a3629e7276a75 P 1dda19a58ad4407ea8b3132be6584367: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260619 02:50:48.816023 24095 tablet_bootstrap.cc:492] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19: No bootstrap required, opened a new log
I20260619 02:50:48.816231 24095 ts_tablet_manager.cc:1397] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.003s
I20260619 02:50:48.823112 24095 raft_consensus.cc:348] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dda19a58ad4407ea8b3132be6584367" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 41497 } } peers { permanent_uuid: "232164e4fda94b69beaabf55732722a8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 44653 } } peers { permanent_uuid: "522553fed19e486a92da24d0c7fb1e19" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37865 } }
I20260619 02:50:48.823477 24095 raft_consensus.cc:374] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:48.823585 24095 raft_consensus.cc:729] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 522553fed19e486a92da24d0c7fb1e19, State: Initialized, Role: FOLLOWER
I20260619 02:50:48.824162 24095 consensus_queue.cc:260] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dda19a58ad4407ea8b3132be6584367" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 41497 } } peers { permanent_uuid: "232164e4fda94b69beaabf55732722a8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 44653 } } peers { permanent_uuid: "522553fed19e486a92da24d0c7fb1e19" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37865 } }
I20260619 02:50:48.824716 24093 raft_consensus.cc:348] T a14a258ed6f74038a34a3629e7276a75 P 232164e4fda94b69beaabf55732722a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dda19a58ad4407ea8b3132be6584367" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 41497 } } peers { permanent_uuid: "232164e4fda94b69beaabf55732722a8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 44653 } } peers { permanent_uuid: "522553fed19e486a92da24d0c7fb1e19" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37865 } }
I20260619 02:50:48.825184 23945 heartbeater.cc:503] Master 127.18.134.62:38565 was elected leader, sending a full tablet report...
I20260619 02:50:48.825207 24093 raft_consensus.cc:374] T a14a258ed6f74038a34a3629e7276a75 P 232164e4fda94b69beaabf55732722a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:48.825286 24093 raft_consensus.cc:729] T a14a258ed6f74038a34a3629e7276a75 P 232164e4fda94b69beaabf55732722a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 232164e4fda94b69beaabf55732722a8, State: Initialized, Role: FOLLOWER
I20260619 02:50:48.825141 24094 raft_consensus.cc:348] T a14a258ed6f74038a34a3629e7276a75 P 1dda19a58ad4407ea8b3132be6584367 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dda19a58ad4407ea8b3132be6584367" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 41497 } } peers { permanent_uuid: "232164e4fda94b69beaabf55732722a8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 44653 } } peers { permanent_uuid: "522553fed19e486a92da24d0c7fb1e19" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37865 } }
I20260619 02:50:48.825524 24094 raft_consensus.cc:374] T a14a258ed6f74038a34a3629e7276a75 P 1dda19a58ad4407ea8b3132be6584367 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:48.825611 24094 raft_consensus.cc:729] T a14a258ed6f74038a34a3629e7276a75 P 1dda19a58ad4407ea8b3132be6584367 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dda19a58ad4407ea8b3132be6584367, State: Initialized, Role: FOLLOWER
I20260619 02:50:48.825809 24095 ts_tablet_manager.cc:1428] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19: Time spent starting tablet: real 0.009s user 0.009s sys 0.001s
I20260619 02:50:48.826085 24093 consensus_queue.cc:260] T a14a258ed6f74038a34a3629e7276a75 P 232164e4fda94b69beaabf55732722a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dda19a58ad4407ea8b3132be6584367" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 41497 } } peers { permanent_uuid: "232164e4fda94b69beaabf55732722a8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 44653 } } peers { permanent_uuid: "522553fed19e486a92da24d0c7fb1e19" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37865 } }
I20260619 02:50:48.826097 24094 consensus_queue.cc:260] T a14a258ed6f74038a34a3629e7276a75 P 1dda19a58ad4407ea8b3132be6584367 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dda19a58ad4407ea8b3132be6584367" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 41497 } } peers { permanent_uuid: "232164e4fda94b69beaabf55732722a8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 44653 } } peers { permanent_uuid: "522553fed19e486a92da24d0c7fb1e19" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37865 } }
I20260619 02:50:48.827549 23815 heartbeater.cc:503] Master 127.18.134.62:38565 was elected leader, sending a full tablet report...
I20260619 02:50:48.827775 24094 ts_tablet_manager.cc:1428] T a14a258ed6f74038a34a3629e7276a75 P 1dda19a58ad4407ea8b3132be6584367: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260619 02:50:48.828069 24093 ts_tablet_manager.cc:1428] T a14a258ed6f74038a34a3629e7276a75 P 232164e4fda94b69beaabf55732722a8: Time spent starting tablet: real 0.014s user 0.011s sys 0.001s
I20260619 02:50:48.828528 24075 heartbeater.cc:503] Master 127.18.134.62:38565 was elected leader, sending a full tablet report...
W20260619 02:50:48.909017 23816 tablet.cc:2369] T a14a258ed6f74038a34a3629e7276a75 P 232164e4fda94b69beaabf55732722a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:50:48.965773 24099 raft_consensus.cc:482] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:50:48.966086 24099 raft_consensus.cc:504] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dda19a58ad4407ea8b3132be6584367" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 41497 } } peers { permanent_uuid: "232164e4fda94b69beaabf55732722a8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 44653 } } peers { permanent_uuid: "522553fed19e486a92da24d0c7fb1e19" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37865 } }
I20260619 02:50:48.967613 24099 leader_election.cc:288] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1dda19a58ad4407ea8b3132be6584367 (127.18.134.3:41497), 232164e4fda94b69beaabf55732722a8 (127.18.134.1:44653)
I20260619 02:50:48.970541 24091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:48.968064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:45993 (local address 127.18.134.1:44653)
0619 02:50:48.968447 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:48.968454 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:48.968607 (+ 153us) server_negotiation.cc:407] Connection header received
0619 02:50:48.968786 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:48.968791 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:48.968840 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:48.968970 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:48.969954 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:48.969958 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:48.969961 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:50:48.969976 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:48.970008 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:50:48.970020 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:48.970127 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:48.970212 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 02:50:48.970269 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271}
I20260619 02:50:48.970551 24089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:48.968028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:53313 (local address 127.18.134.3:41497)
0619 02:50:48.968297 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:48.968305 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:48.968935 (+ 630us) server_negotiation.cc:407] Connection header received
0619 02:50:48.969029 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:48.969033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:48.969077 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:48.969154 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:48.970008 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:48.970014 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:48.970018 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:50:48.970033 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:48.970060 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:50:48.970071 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:48.970178 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:48.970255 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 02:50:48.970312 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260619 02:50:48.970760 24102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:48.967900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:44653 (local address 127.18.134.2:45993)
0619 02:50:48.968494 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:48.968516 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:48.968651 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:48.969013 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:48.969017 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:48.969041 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:48.969708 (+ 667us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:50:48.969711 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:48.969724 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:50:48.969763 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:48.970111 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:48.970116 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:48.970121 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:50:48.970219 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 02:50:48.970269 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":345,"threads_started":1}
I20260619 02:50:48.970834 24103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:48.967927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:41497 (local address 127.18.134.2:53313)
0619 02:50:48.968834 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:48.968850 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:48.968935 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:48.969197 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:48.969202 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:48.969223 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:48.969829 (+ 606us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:50:48.969831 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:48.969844 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:50:48.969877 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:48.970154 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:48.970158 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:48.970162 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:50:48.970284 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 02:50:48.970330 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":813,"thread_start_us":615,"threads_started":1}
I20260619 02:50:48.971671 24030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a14a258ed6f74038a34a3629e7276a75" candidate_uuid: "522553fed19e486a92da24d0c7fb1e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dda19a58ad4407ea8b3132be6584367" is_pre_election: true
I20260619 02:50:48.971664 23770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a14a258ed6f74038a34a3629e7276a75" candidate_uuid: "522553fed19e486a92da24d0c7fb1e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "232164e4fda94b69beaabf55732722a8" is_pre_election: true
I20260619 02:50:48.972136 24030 raft_consensus.cc:2454] T a14a258ed6f74038a34a3629e7276a75 P 1dda19a58ad4407ea8b3132be6584367 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522553fed19e486a92da24d0c7fb1e19 in term 0.
I20260619 02:50:48.972136 23770 raft_consensus.cc:2454] T a14a258ed6f74038a34a3629e7276a75 P 232164e4fda94b69beaabf55732722a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 522553fed19e486a92da24d0c7fb1e19 in term 0.
I20260619 02:50:48.972821 23835 leader_election.cc:302] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 232164e4fda94b69beaabf55732722a8, 522553fed19e486a92da24d0c7fb1e19; no voters:
I20260619 02:50:48.973210 24099 raft_consensus.cc:2781] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:50:48.973359 24099 raft_consensus.cc:482] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:50:48.973479 24099 raft_consensus.cc:3037] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:48.975602 24099 raft_consensus.cc:504] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dda19a58ad4407ea8b3132be6584367" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 41497 } } peers { permanent_uuid: "232164e4fda94b69beaabf55732722a8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 44653 } } peers { permanent_uuid: "522553fed19e486a92da24d0c7fb1e19" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37865 } }
I20260619 02:50:48.976212 24099 leader_election.cc:288] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19 [CANDIDATE]: Term 1 election: Requested vote from peers 1dda19a58ad4407ea8b3132be6584367 (127.18.134.3:41497), 232164e4fda94b69beaabf55732722a8 (127.18.134.1:44653)
I20260619 02:50:48.976572 24030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a14a258ed6f74038a34a3629e7276a75" candidate_uuid: "522553fed19e486a92da24d0c7fb1e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1dda19a58ad4407ea8b3132be6584367"
I20260619 02:50:48.976728 23770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a14a258ed6f74038a34a3629e7276a75" candidate_uuid: "522553fed19e486a92da24d0c7fb1e19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "232164e4fda94b69beaabf55732722a8"
I20260619 02:50:48.976776 24030 raft_consensus.cc:3037] T a14a258ed6f74038a34a3629e7276a75 P 1dda19a58ad4407ea8b3132be6584367 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:48.976961 23770 raft_consensus.cc:3037] T a14a258ed6f74038a34a3629e7276a75 P 232164e4fda94b69beaabf55732722a8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:48.979943 24030 raft_consensus.cc:2454] T a14a258ed6f74038a34a3629e7276a75 P 1dda19a58ad4407ea8b3132be6584367 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522553fed19e486a92da24d0c7fb1e19 in term 1.
I20260619 02:50:48.979959 23770 raft_consensus.cc:2454] T a14a258ed6f74038a34a3629e7276a75 P 232164e4fda94b69beaabf55732722a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 522553fed19e486a92da24d0c7fb1e19 in term 1.
I20260619 02:50:48.980386 23834 leader_election.cc:302] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dda19a58ad4407ea8b3132be6584367, 522553fed19e486a92da24d0c7fb1e19; no voters:
I20260619 02:50:48.980693 24099 raft_consensus.cc:2781] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:50:48.981123 24099 raft_consensus.cc:686] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19 [term 1 LEADER]: Becoming Leader. State: Replica: 522553fed19e486a92da24d0c7fb1e19, State: Running, Role: LEADER
I20260619 02:50:48.981562 24099 consensus_queue.cc:237] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dda19a58ad4407ea8b3132be6584367" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 41497 } } peers { permanent_uuid: "232164e4fda94b69beaabf55732722a8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 44653 } } peers { permanent_uuid: "522553fed19e486a92da24d0c7fb1e19" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37865 } }
W20260619 02:50:48.983831 24076 tablet.cc:2369] T a14a258ed6f74038a34a3629e7276a75 P 1dda19a58ad4407ea8b3132be6584367: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:50:48.986186 23628 catalog_manager.cc:5495] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 522553fed19e486a92da24d0c7fb1e19 (127.18.134.2). New cstate: current_term: 1 leader_uuid: "522553fed19e486a92da24d0c7fb1e19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1dda19a58ad4407ea8b3132be6584367" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 41497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "232164e4fda94b69beaabf55732722a8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 44653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "522553fed19e486a92da24d0c7fb1e19" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37865 } health_report { overall_health: HEALTHY } } }
I20260619 02:50:49.028301 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:49.026266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36952 (local address 127.18.134.2:37865)
0619 02:50:49.026460 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:49.026466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:49.026548 (+ 82us) server_negotiation.cc:407] Connection header received
0619 02:50:49.026791 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:49.026795 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:49.026873 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:49.026991 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:49.027761 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:49.027767 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:49.027772 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:50:49.027789 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:49.027817 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:50:49.027828 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:49.027938 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:49.028016 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 02:50:49.028070 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 02:50:49.035738 24030 raft_consensus.cc:1261] T a14a258ed6f74038a34a3629e7276a75 P 1dda19a58ad4407ea8b3132be6584367 [term 1 FOLLOWER]: Refusing update from remote peer 522553fed19e486a92da24d0c7fb1e19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:50:49.035737 23770 raft_consensus.cc:1261] T a14a258ed6f74038a34a3629e7276a75 P 232164e4fda94b69beaabf55732722a8 [term 1 FOLLOWER]: Refusing update from remote peer 522553fed19e486a92da24d0c7fb1e19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:50:49.036607 24104 consensus_queue.cc:1035] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "232164e4fda94b69beaabf55732722a8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 44653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:50:49.036907 24099 consensus_queue.cc:1035] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1dda19a58ad4407ea8b3132be6584367" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 41497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:50:49.061172 23629 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:50:49.061458 23629 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:50:49.065904 23629 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 58a32f13e7034cbb8cf2242ad6a81e2d: Sending DeleteTablet for 3 replicas of tablet a14a258ed6f74038a34a3629e7276a75
I20260619 02:50:49.067101 24010 tablet_service.cc:1514] Processing DeleteTablet for tablet a14a258ed6f74038a34a3629e7276a75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:50:49 UTC) from {username='slave'} at 127.0.0.1:60900
I20260619 02:50:49.067350 23750 tablet_service.cc:1514] Processing DeleteTablet for tablet a14a258ed6f74038a34a3629e7276a75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:50:49 UTC) from {username='slave'} at 127.0.0.1:39642
I20260619 02:50:49.068007 24117 tablet_replica.cc:330] T a14a258ed6f74038a34a3629e7276a75 P 1dda19a58ad4407ea8b3132be6584367: stopping tablet replica
I20260619 02:50:49.067953 23880 tablet_service.cc:1514] Processing DeleteTablet for tablet a14a258ed6f74038a34a3629e7276a75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:50:49 UTC) from {username='slave'} at 127.0.0.1:36946
I20260619 02:50:49.068270 24118 tablet_replica.cc:330] T a14a258ed6f74038a34a3629e7276a75 P 232164e4fda94b69beaabf55732722a8: stopping tablet replica
I20260619 02:50:49.068420 24117 raft_consensus.cc:2229] T a14a258ed6f74038a34a3629e7276a75 P 1dda19a58ad4407ea8b3132be6584367 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:50:49.068766 24118 raft_consensus.cc:2229] T a14a258ed6f74038a34a3629e7276a75 P 232164e4fda94b69beaabf55732722a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:50:49.068944 24117 raft_consensus.cc:2258] T a14a258ed6f74038a34a3629e7276a75 P 1dda19a58ad4407ea8b3132be6584367 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:50:49.069330 24118 raft_consensus.cc:2258] T a14a258ed6f74038a34a3629e7276a75 P 232164e4fda94b69beaabf55732722a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:50:49.071000 24117 ts_tablet_manager.cc:1905] T a14a258ed6f74038a34a3629e7276a75 P 1dda19a58ad4407ea8b3132be6584367: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:50:49.071578 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 23688
I20260619 02:50:49.072973 24118 ts_tablet_manager.cc:1905] T a14a258ed6f74038a34a3629e7276a75 P 232164e4fda94b69beaabf55732722a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:50:49.078320 24117 ts_tablet_manager.cc:1918] T a14a258ed6f74038a34a3629e7276a75 P 1dda19a58ad4407ea8b3132be6584367: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:50:49.078495 24117 log.cc:1196] T a14a258ed6f74038a34a3629e7276a75 P 1dda19a58ad4407ea8b3132be6584367: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-2/wal/wals/a14a258ed6f74038a34a3629e7276a75
I20260619 02:50:49.079051 24117 ts_tablet_manager.cc:1939] T a14a258ed6f74038a34a3629e7276a75 P 1dda19a58ad4407ea8b3132be6584367: Deleting consensus metadata
I20260619 02:50:49.080102 24119 tablet_replica.cc:330] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19: stopping tablet replica
I20260619 02:50:49.080474 24118 ts_tablet_manager.cc:1918] T a14a258ed6f74038a34a3629e7276a75 P 232164e4fda94b69beaabf55732722a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:50:49.080545 24119 raft_consensus.cc:2229] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:50:49.080955 23614 catalog_manager.cc:4841] TS 1dda19a58ad4407ea8b3132be6584367 (127.18.134.3:41497): tablet a14a258ed6f74038a34a3629e7276a75 (table test-table [id=874e0452fd054e21b549a5fb4056c7a4]) successfully deleted
I20260619 02:50:49.081209 24119 raft_consensus.cc:2258] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:50:49.081394 24118 log.cc:1196] T a14a258ed6f74038a34a3629e7276a75 P 232164e4fda94b69beaabf55732722a8: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-0/wal/wals/a14a258ed6f74038a34a3629e7276a75
I20260619 02:50:49.082016 24118 ts_tablet_manager.cc:1939] T a14a258ed6f74038a34a3629e7276a75 P 232164e4fda94b69beaabf55732722a8: Deleting consensus metadata
I20260619 02:50:49.082894 24119 ts_tablet_manager.cc:1905] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:50:49.083827 23615 catalog_manager.cc:4841] TS 232164e4fda94b69beaabf55732722a8 (127.18.134.1:44653): tablet a14a258ed6f74038a34a3629e7276a75 (table test-table [id=874e0452fd054e21b549a5fb4056c7a4]) successfully deleted
I20260619 02:50:49.087941 24119 ts_tablet_manager.cc:1918] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:50:49.088074 24119 log.cc:1196] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781837410515220-18968-0/minicluster-data/ts-1/wal/wals/a14a258ed6f74038a34a3629e7276a75
I20260619 02:50:49.088577 24119 ts_tablet_manager.cc:1939] T a14a258ed6f74038a34a3629e7276a75 P 522553fed19e486a92da24d0c7fb1e19: Deleting consensus metadata
I20260619 02:50:49.089273 24091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:49.073457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39652 (local address 127.18.134.1:44653)
0619 02:50:49.073620 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:49.073626 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:49.087189 (+ 13563us) server_negotiation.cc:407] Connection header received
0619 02:50:49.087348 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:49.087362 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:49.087461 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:49.087571 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:49.088576 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:49.088582 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:49.088587 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:50:49.088607 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:49.088644 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:50:49.088659 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:49.088783 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:49.088905 (+ 122us) server_negotiation.cc:299] Negotiation successful
0619 02:50:49.088969 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 02:50:49.089941 23810 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:49.090305 23613 catalog_manager.cc:4841] TS 522553fed19e486a92da24d0c7fb1e19 (127.18.134.2:37865): tablet a14a258ed6f74038a34a3629e7276a75 (table test-table [id=874e0452fd054e21b549a5fb4056c7a4]) successfully deleted
I20260619 02:50:49.175278 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 23688
I20260619 02:50:49.195883 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 23818
I20260619 02:50:49.198781 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:49.196665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36962 (local address 127.18.134.2:37865)
0619 02:50:49.196819 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:49.196829 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:49.196929 (+ 100us) server_negotiation.cc:407] Connection header received
0619 02:50:49.197115 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:49.197119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:49.197196 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:49.197307 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:49.198147 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:49.198153 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:49.198158 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:50:49.198176 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:49.198203 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:50:49.198214 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:49.198329 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:49.198427 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 02:50:49.198489 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 02:50:49.199399 23940 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:49.303687 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 23818
I20260619 02:50:49.317379 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 23948
I20260619 02:50:49.319939 24089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:49.318059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60912 (local address 127.18.134.3:41497)
0619 02:50:49.318216 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:49.318220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:49.318280 (+ 60us) server_negotiation.cc:407] Connection header received
0619 02:50:49.318440 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:49.318444 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:49.318517 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:49.318599 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:49.319408 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:49.319412 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:49.319416 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:50:49.319430 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:49.319460 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:50:49.319471 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:49.319583 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:49.319674 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 02:50:49.319718 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 02:50:49.320533 24070 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:49.405031 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 23948
I20260619 02:50:49.417333 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 23598
I20260619 02:50:49.418444 23659 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:49.519009 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 23598
2026-06-19T02:50:49Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2553 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260619 02:50:49.554513 18968 mini_oidc.cc:152] Starting JWKS server
I20260619 02:50:49.556916 18968 webserver.cc:460] Webserver started at https://127.0.0.1:46197/ using document root <none> and password file <none>
I20260619 02:50:49.557195 18968 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 02:50:49.557758 18968 webserver.cc:460] Webserver started at http://127.0.0.1:33789/ using document root <none> and password file <none>
2026-06-19T02:50:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:50:49Z Disabled control of system clock
I20260619 02:50:49.582963 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:39329
--webserver_interface=127.18.134.62
--webserver_port=0
--builtin_ntp_servers=127.18.134.20:45113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:39329
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46197/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837410515220-18968-0/testchainca.cert with env {}
W20260619 02:50:49.887279 24130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:49.887550 24130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:49.887662 24130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:49.894955 24130 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:50:49.895071 24130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:49.895136 24130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:49.895210 24130 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:50:49.895254 24130 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 02:50:49.895331 24130 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 02:50:49.895387 24130 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46197/jwks.json
I20260619 02:50:49.905213 24130 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:45113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:39329
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:39329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46197/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837410515220-18968-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:49.906490 24130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:49.907765 24130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:49.918748 24135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:49.918821 24138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:49.919060 24136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:49.919715 24130 server_base.cc:1034] running on GCE node
I20260619 02:50:49.920632 24130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:49.922006 24130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:49.923228 24130 hybrid_clock.cc:648] HybridClock initialized: now 1781837449923178 us; error 35 us; skew 500 ppm
I20260619 02:50:49.923578 24130 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:50:49.925776 24130 webserver.cc:460] Webserver started at http://127.18.134.62:38385/ using document root <none> and password file <none>
I20260619 02:50:49.926427 24130 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:49.926530 24130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:49.926843 24130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:49.929584 24130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "a1c8ef4d022747bd8801083c6cd7ce65"
format_stamp: "Formatted at 2026-06-19 02:50:49 on dist-test-slave-9mr0"
server_key: "c1dcd1208e9925f305fcb70bb4e724cf"
server_key_iv: "275727bdfd628efd52a3e5b814e4d78c"
server_key_version: "encryptionkey@0"
I20260619 02:50:49.930322 24130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "a1c8ef4d022747bd8801083c6cd7ce65"
format_stamp: "Formatted at 2026-06-19 02:50:49 on dist-test-slave-9mr0"
server_key: "c1dcd1208e9925f305fcb70bb4e724cf"
server_key_iv: "275727bdfd628efd52a3e5b814e4d78c"
server_key_version: "encryptionkey@0"
I20260619 02:50:49.935251 24130 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260619 02:50:49.938534 24144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:49.939894 24130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:50:49.940083 24130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "a1c8ef4d022747bd8801083c6cd7ce65"
format_stamp: "Formatted at 2026-06-19 02:50:49 on dist-test-slave-9mr0"
server_key: "c1dcd1208e9925f305fcb70bb4e724cf"
server_key_iv: "275727bdfd628efd52a3e5b814e4d78c"
server_key_version: "encryptionkey@0"
I20260619 02:50:49.940274 24130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:49.974020 24130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:49.974905 24130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:49.975194 24130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:49.992456 24130 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:39329
I20260619 02:50:49.992486 24195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:39329 every 8 connection(s)
I20260619 02:50:49.993758 24130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:50:49.997357 24196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:49.999704 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 24130
I20260619 02:50:49.999917 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:50:50.000638 18968 external_mini_cluster.cc:1428] Setting key ebf6fb0aa4b30fd92fd69d219ecd0ee5
I20260619 02:50:50.008755 24196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1c8ef4d022747bd8801083c6cd7ce65: Bootstrap starting.
I20260619 02:50:50.011960 24196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1c8ef4d022747bd8801083c6cd7ce65: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:50.013223 24196 log.cc:824] T 00000000000000000000000000000000 P a1c8ef4d022747bd8801083c6cd7ce65: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:50.013914 24199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:50.002575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50620 (local address 127.18.134.62:39329)
0619 02:50:50.003584 (+ 1009us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:50.003596 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:50.003635 (+ 39us) server_negotiation.cc:407] Connection header received
0619 02:50:50.005197 (+ 1562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:50.005220 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:50.005695 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:50.006061 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:50.006684 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:50.007926 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:50.009239 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:50.009892 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:50.010073 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:50.011284 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:50.011291 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:50.011305 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:50:50.011826 (+ 521us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:50.011904 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:50:50.011924 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:50.012028 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:50.012260 (+ 232us) server_negotiation.cc:299] Negotiation successful
0619 02:50:50.012611 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":335,"threads_started":1}
I20260619 02:50:50.016470 24196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1c8ef4d022747bd8801083c6cd7ce65: No bootstrap required, opened a new log
I20260619 02:50:50.017692 18968 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:50:50.024154 24196 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1c8ef4d022747bd8801083c6cd7ce65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c8ef4d022747bd8801083c6cd7ce65" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39329 } }
I20260619 02:50:50.024550 24196 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1c8ef4d022747bd8801083c6cd7ce65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:50.024643 24196 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1c8ef4d022747bd8801083c6cd7ce65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1c8ef4d022747bd8801083c6cd7ce65, State: Initialized, Role: FOLLOWER
I20260619 02:50:50.025197 24196 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1c8ef4d022747bd8801083c6cd7ce65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c8ef4d022747bd8801083c6cd7ce65" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39329 } }
I20260619 02:50:50.025403 24196 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1c8ef4d022747bd8801083c6cd7ce65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:50:50.025523 24196 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1c8ef4d022747bd8801083c6cd7ce65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:50:50.025672 24196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1c8ef4d022747bd8801083c6cd7ce65 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:50.027832 24196 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1c8ef4d022747bd8801083c6cd7ce65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c8ef4d022747bd8801083c6cd7ce65" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39329 } }
I20260619 02:50:50.028311 24196 leader_election.cc:302] T 00000000000000000000000000000000 P a1c8ef4d022747bd8801083c6cd7ce65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1c8ef4d022747bd8801083c6cd7ce65; no voters:
I20260619 02:50:50.028811 24196 leader_election.cc:288] T 00000000000000000000000000000000 P a1c8ef4d022747bd8801083c6cd7ce65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:50:50.028941 24201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1c8ef4d022747bd8801083c6cd7ce65 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:50:50.029686 24201 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1c8ef4d022747bd8801083c6cd7ce65 [term 1 LEADER]: Becoming Leader. State: Replica: a1c8ef4d022747bd8801083c6cd7ce65, State: Running, Role: LEADER
I20260619 02:50:50.030155 24201 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1c8ef4d022747bd8801083c6cd7ce65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c8ef4d022747bd8801083c6cd7ce65" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39329 } }
I20260619 02:50:50.030773 24196 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1c8ef4d022747bd8801083c6cd7ce65 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:50:50.032838 24202 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1c8ef4d022747bd8801083c6cd7ce65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1c8ef4d022747bd8801083c6cd7ce65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c8ef4d022747bd8801083c6cd7ce65" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39329 } } }
I20260619 02:50:50.032831 24203 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1c8ef4d022747bd8801083c6cd7ce65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1c8ef4d022747bd8801083c6cd7ce65. Latest consensus state: current_term: 1 leader_uuid: "a1c8ef4d022747bd8801083c6cd7ce65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c8ef4d022747bd8801083c6cd7ce65" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39329 } } }
I20260619 02:50:50.033754 24203 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1c8ef4d022747bd8801083c6cd7ce65 [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:50.033749 24202 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1c8ef4d022747bd8801083c6cd7ce65 [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:50.035133 24209 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:50:50.039937 24209 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:50:50.049177 24209 catalog_manager.cc:1347] Generated new cluster ID: c94cfad789ca4403ae73c7342b2166d3
I20260619 02:50:50.049302 24209 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:50:50.064023 24209 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:50:50.065732 24209 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:50:50.077956 24209 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1c8ef4d022747bd8801083c6cd7ce65: Generated new TSK 0
I20260619 02:50:50.079016 24209 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 02:50:50.128024 24226 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260619 02:50:50.128198 24226 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 02:50:50.128875 24199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:50.100461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50628 (local address 127.18.134.62:39329)
0619 02:50:50.100700 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:50.100709 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:50.101181 (+ 472us) server_negotiation.cc:407] Connection header received
0619 02:50:50.101498 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:50.101502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:50.101580 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:50.101706 (+ 126us) server_negotiation.cc:226] Negotiated authn=JWT
0619 02:50:50.102317 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:50.103180 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:50.105193 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:50.105624 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:50.105712 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:50.106324 (+ 612us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 02:50:50.128363 (+ 22039us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46197/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 02:50:50.128662 (+ 299us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50628: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46197/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":65}
W20260619 02:50:50.128991 24199 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50628: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46197/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260619 02:50:50.130762 24225 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:50:50.100396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39329 (local address 127.0.0.1:50628)
0619 02:50:50.101068 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:50.101092 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:50.101266 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:50.101753 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:50.101758 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:50.101770 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0619 02:50:50.102150 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:50.102162 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:50.103315 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:50.103321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:50.105049 (+ 1728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:50.105063 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:50.105784 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:50.105791 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:50.105982 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:50.106059 (+ 77us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 02:50:50.128644 (+ 22585us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46197/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 02:50:50.129000 (+ 356us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.134.62:39329: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46197/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":320,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.18.134.62:39329: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46197/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260619 02:50:50.132346 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 24130
I20260619 02:50:50.134045 24191 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:50.224438 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 24130
2026-06-19T02:50:50Z chronyd exiting
I20260619 02:50:50.270556 18968 test_util.cc:175] -----------------------------------------------
I20260619 02:50:50.270740 18968 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781837410515220-18968-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (720 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260619 02:50:50.274500 18968 mini_oidc.cc:152] Starting JWKS server
I20260619 02:50:50.276286 18968 webserver.cc:460] Webserver started at https://127.0.0.1:43569/ using document root <none> and password file <none>
I20260619 02:50:50.276469 18968 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 02:50:50.277101 18968 webserver.cc:460] Webserver started at http://127.0.0.1:36941/ using document root <none> and password file <none>
2026-06-19T02:50:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:50:50Z Disabled control of system clock
I20260619 02:50:50.300115 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:32873
--webserver_interface=127.18.134.62
--webserver_port=0
--builtin_ntp_servers=127.18.134.20:39487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:32873
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43569/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837410515220-18968-0/testchainca.cert with env {}
W20260619 02:50:50.608912 24233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:50.609230 24233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:50.609378 24233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:50.616685 24233 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:50:50.616833 24233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:50.616907 24233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:50.616992 24233 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:50:50.617058 24233 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 02:50:50.617153 24233 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 02:50:50.617213 24233 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43569/jwks.json
I20260619 02:50:50.627537 24233 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:39487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:32873
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:32873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43569/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837410515220-18968-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:50.628880 24233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:50.630232 24233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:50.641788 24238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:50.641819 24239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:50.641853 24241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:50.642529 24233 server_base.cc:1034] running on GCE node
I20260619 02:50:50.643343 24233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:50.644815 24233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:50.646036 24233 hybrid_clock.cc:648] HybridClock initialized: now 1781837450645964 us; error 68 us; skew 500 ppm
I20260619 02:50:50.646420 24233 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:50:50.648732 24233 webserver.cc:460] Webserver started at http://127.18.134.62:38609/ using document root <none> and password file <none>
I20260619 02:50:50.649418 24233 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:50.649513 24233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:50.649814 24233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:50.652639 24233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "616bf7b5128b4ac1b17d98022972062d"
format_stamp: "Formatted at 2026-06-19 02:50:50 on dist-test-slave-9mr0"
server_key: "d12c33dc3d476779f68821832015130d"
server_key_iv: "885b2e60fd861308aa36993737b53119"
server_key_version: "encryptionkey@0"
I20260619 02:50:50.653400 24233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "616bf7b5128b4ac1b17d98022972062d"
format_stamp: "Formatted at 2026-06-19 02:50:50 on dist-test-slave-9mr0"
server_key: "d12c33dc3d476779f68821832015130d"
server_key_iv: "885b2e60fd861308aa36993737b53119"
server_key_version: "encryptionkey@0"
I20260619 02:50:50.658703 24233 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260619 02:50:50.662165 24247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:50.663592 24233 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260619 02:50:50.663753 24233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "616bf7b5128b4ac1b17d98022972062d"
format_stamp: "Formatted at 2026-06-19 02:50:50 on dist-test-slave-9mr0"
server_key: "d12c33dc3d476779f68821832015130d"
server_key_iv: "885b2e60fd861308aa36993737b53119"
server_key_version: "encryptionkey@0"
I20260619 02:50:50.663960 24233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:50.696952 24233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:50.697811 24233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:50.698105 24233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:50.715528 24233 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:32873
I20260619 02:50:50.715570 24298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:32873 every 8 connection(s)
I20260619 02:50:50.716851 24233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:50:50.717527 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 24233
I20260619 02:50:50.717707 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:50:50.718437 18968 external_mini_cluster.cc:1428] Setting key fb0619f6176d4d53dca20ba90a3f3927
I20260619 02:50:50.721022 24299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:50.731951 24302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:50.720330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42892 (local address 127.18.134.62:32873)
0619 02:50:50.721284 (+ 954us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:50.721296 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:50.721337 (+ 41us) server_negotiation.cc:407] Connection header received
0619 02:50:50.722893 (+ 1556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:50.722914 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:50.723378 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:50.723709 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:50.724292 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:50.725620 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:50.727448 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:50.728127 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:50.728266 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:50.729558 (+ 1292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:50.729566 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:50.729581 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:50:50.730187 (+ 606us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:50.730275 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:50:50.730296 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:50.730395 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:50.730612 (+ 217us) server_negotiation.cc:299] Negotiation successful
0619 02:50:50.731010 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":307,"threads_started":1}
I20260619 02:50:50.732698 24299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 616bf7b5128b4ac1b17d98022972062d: Bootstrap starting.
I20260619 02:50:50.735853 18968 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:50:50.736279 24299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 616bf7b5128b4ac1b17d98022972062d: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:50.737291 24299 log.cc:824] T 00000000000000000000000000000000 P 616bf7b5128b4ac1b17d98022972062d: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:50.740142 24299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 616bf7b5128b4ac1b17d98022972062d: No bootstrap required, opened a new log
I20260619 02:50:50.747339 24299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 616bf7b5128b4ac1b17d98022972062d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "616bf7b5128b4ac1b17d98022972062d" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32873 } }
I20260619 02:50:50.747733 24299 raft_consensus.cc:374] T 00000000000000000000000000000000 P 616bf7b5128b4ac1b17d98022972062d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:50.747828 24299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 616bf7b5128b4ac1b17d98022972062d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 616bf7b5128b4ac1b17d98022972062d, State: Initialized, Role: FOLLOWER
I20260619 02:50:50.748364 24299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 616bf7b5128b4ac1b17d98022972062d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "616bf7b5128b4ac1b17d98022972062d" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32873 } }
I20260619 02:50:50.748585 24299 raft_consensus.cc:388] T 00000000000000000000000000000000 P 616bf7b5128b4ac1b17d98022972062d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:50:50.748704 24299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 616bf7b5128b4ac1b17d98022972062d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:50:50.748853 24299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 616bf7b5128b4ac1b17d98022972062d [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:50.751070 24299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 616bf7b5128b4ac1b17d98022972062d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "616bf7b5128b4ac1b17d98022972062d" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32873 } }
I20260619 02:50:50.751569 24299 leader_election.cc:302] T 00000000000000000000000000000000 P 616bf7b5128b4ac1b17d98022972062d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 616bf7b5128b4ac1b17d98022972062d; no voters:
I20260619 02:50:50.752166 24299 leader_election.cc:288] T 00000000000000000000000000000000 P 616bf7b5128b4ac1b17d98022972062d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:50:50.752312 24304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 616bf7b5128b4ac1b17d98022972062d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:50:50.753257 24304 raft_consensus.cc:686] T 00000000000000000000000000000000 P 616bf7b5128b4ac1b17d98022972062d [term 1 LEADER]: Becoming Leader. State: Replica: 616bf7b5128b4ac1b17d98022972062d, State: Running, Role: LEADER
I20260619 02:50:50.753786 24304 consensus_queue.cc:237] T 00000000000000000000000000000000 P 616bf7b5128b4ac1b17d98022972062d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "616bf7b5128b4ac1b17d98022972062d" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32873 } }
I20260619 02:50:50.754426 24299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 616bf7b5128b4ac1b17d98022972062d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:50:50.756836 24305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 616bf7b5128b4ac1b17d98022972062d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "616bf7b5128b4ac1b17d98022972062d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "616bf7b5128b4ac1b17d98022972062d" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32873 } } }
I20260619 02:50:50.756872 24306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 616bf7b5128b4ac1b17d98022972062d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 616bf7b5128b4ac1b17d98022972062d. Latest consensus state: current_term: 1 leader_uuid: "616bf7b5128b4ac1b17d98022972062d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "616bf7b5128b4ac1b17d98022972062d" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32873 } } }
I20260619 02:50:50.757239 24306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 616bf7b5128b4ac1b17d98022972062d [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:50.757216 24305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 616bf7b5128b4ac1b17d98022972062d [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:50.758846 24312 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:50:50.764047 24312 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:50:50.772959 24312 catalog_manager.cc:1347] Generated new cluster ID: b623ada993b94898b527970978761a44
I20260619 02:50:50.773072 24312 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:50:50.816278 24312 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:50:50.818089 24312 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:50:50.831809 24312 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 616bf7b5128b4ac1b17d98022972062d: Generated new TSK 0
I20260619 02:50:50.832842 24312 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 02:50:50.871712 24329 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260619 02:50:50.871824 24329 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 02:50:50.873111 24302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:50.852811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42906 (local address 127.18.134.62:32873)
0619 02:50:50.852986 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:50.852993 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:50.853464 (+ 471us) server_negotiation.cc:407] Connection header received
0619 02:50:50.853684 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:50.853690 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:50.853798 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:50.853953 (+ 155us) server_negotiation.cc:226] Negotiated authn=JWT
0619 02:50:50.854436 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:50.855649 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:50.856949 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:50.857640 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:50.857766 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:50.858299 (+ 533us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 02:50:50.872409 (+ 14110us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43569/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 02:50:50.872806 (+ 397us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42906: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43569/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":49}
W20260619 02:50:50.873132 24328 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:50:50.852801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:32873 (local address 127.0.0.1:42906)
0619 02:50:50.853378 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:50.853393 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:50.853515 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:50.853950 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:50.853953 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:50.853962 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0619 02:50:50.854234 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:50.854244 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:50.855794 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:50.855798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:50.856814 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:50.856825 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:50.857831 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:50.857836 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:50.857983 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:50.858019 (+ 36us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 02:50:50.872644 (+ 14625us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43569/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 02:50:50.872844 (+ 200us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.134.62:32873: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43569/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":307,"threads_started":1}
W20260619 02:50:50.873234 24302 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42906: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43569/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260619 02:50:50.874934 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 24233
I20260619 02:50:50.875927 24294 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:50.970032 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 24233
2026-06-19T02:50:51Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (746 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260619 02:50:51.020722 18968 mini_oidc.cc:152] Starting JWKS server
I20260619 02:50:51.022518 18968 webserver.cc:460] Webserver started at https://127.0.0.1:35323/ using document root <none> and password file <none>
I20260619 02:50:51.022770 18968 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 02:50:51.023366 18968 webserver.cc:460] Webserver started at http://127.0.0.1:34259/ using document root <none> and password file <none>
2026-06-19T02:50:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:50:51Z Disabled control of system clock
I20260619 02:50:51.045859 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:33475
--webserver_interface=127.18.134.62
--webserver_port=0
--builtin_ntp_servers=127.18.134.20:33633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:33475
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35323/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 02:50:51.348639 24336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:51.348922 24336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:51.349009 24336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:51.356076 24336 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:50:51.356189 24336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:51.356225 24336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:51.356281 24336 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:50:51.356324 24336 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 02:50:51.356381 24336 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 02:50:51.356420 24336 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35323/jwks.json
I20260619 02:50:51.365447 24336 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:33633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:33475
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:33475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35323/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:51.366683 24336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:51.368507 24336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:51.379683 24341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:51.379724 24344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:51.379750 24342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:51.380364 24336 server_base.cc:1034] running on GCE node
I20260619 02:50:51.381114 24336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:51.382481 24336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:51.383675 24336 hybrid_clock.cc:648] HybridClock initialized: now 1781837451383623 us; error 46 us; skew 500 ppm
I20260619 02:50:51.384001 24336 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:50:51.386127 24336 webserver.cc:460] Webserver started at http://127.18.134.62:34781/ using document root <none> and password file <none>
I20260619 02:50:51.386827 24336 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:51.386925 24336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:51.387224 24336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:51.389961 24336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "ca2deb2c9b2d400d9987130737635adf"
format_stamp: "Formatted at 2026-06-19 02:50:51 on dist-test-slave-9mr0"
server_key: "7030092610354eff52afb47208eae830"
server_key_iv: "b70e882b54f26cb0ba8ed237f94c2dd6"
server_key_version: "encryptionkey@0"
I20260619 02:50:51.390781 24336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "ca2deb2c9b2d400d9987130737635adf"
format_stamp: "Formatted at 2026-06-19 02:50:51 on dist-test-slave-9mr0"
server_key: "7030092610354eff52afb47208eae830"
server_key_iv: "b70e882b54f26cb0ba8ed237f94c2dd6"
server_key_version: "encryptionkey@0"
I20260619 02:50:51.395903 24336 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 02:50:51.399338 24350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:51.400718 24336 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:50:51.400871 24336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "ca2deb2c9b2d400d9987130737635adf"
format_stamp: "Formatted at 2026-06-19 02:50:51 on dist-test-slave-9mr0"
server_key: "7030092610354eff52afb47208eae830"
server_key_iv: "b70e882b54f26cb0ba8ed237f94c2dd6"
server_key_version: "encryptionkey@0"
I20260619 02:50:51.401087 24336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:51.417954 24336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:51.418809 24336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:51.419094 24336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:51.435562 24336 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:33475
I20260619 02:50:51.435580 24401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:33475 every 8 connection(s)
I20260619 02:50:51.436875 24336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:50:51.440476 24402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:51.441494 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 24336
I20260619 02:50:51.441673 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:50:51.442243 18968 external_mini_cluster.cc:1428] Setting key 5a1a230c3a1f64d578859e5822c0c21a
I20260619 02:50:51.454180 24402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca2deb2c9b2d400d9987130737635adf: Bootstrap starting.
I20260619 02:50:51.456827 24405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:51.444153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40934 (local address 127.18.134.62:33475)
0619 02:50:51.445219 (+ 1066us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:51.445236 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:51.445293 (+ 57us) server_negotiation.cc:407] Connection header received
0619 02:50:51.446893 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:51.446926 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:51.447558 (+ 632us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:51.447995 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:51.448512 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:51.450324 (+ 1812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:51.451694 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:51.452734 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:51.452926 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:51.453992 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:51.454003 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:51.454022 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:50:51.454730 (+ 708us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:51.454822 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:50:51.454847 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:51.454954 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:51.455188 (+ 234us) server_negotiation.cc:299] Negotiation successful
0619 02:50:51.455518 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":333,"threads_started":1}
I20260619 02:50:51.457904 24402 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca2deb2c9b2d400d9987130737635adf: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:51.458925 24402 log.cc:824] T 00000000000000000000000000000000 P ca2deb2c9b2d400d9987130737635adf: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:51.460541 18968 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:50:51.461704 24402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca2deb2c9b2d400d9987130737635adf: No bootstrap required, opened a new log
I20260619 02:50:51.468943 24402 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca2deb2c9b2d400d9987130737635adf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca2deb2c9b2d400d9987130737635adf" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33475 } }
I20260619 02:50:51.469323 24402 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca2deb2c9b2d400d9987130737635adf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:51.469421 24402 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca2deb2c9b2d400d9987130737635adf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca2deb2c9b2d400d9987130737635adf, State: Initialized, Role: FOLLOWER
I20260619 02:50:51.470008 24402 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca2deb2c9b2d400d9987130737635adf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca2deb2c9b2d400d9987130737635adf" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33475 } }
I20260619 02:50:51.470211 24402 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca2deb2c9b2d400d9987130737635adf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:50:51.470332 24402 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca2deb2c9b2d400d9987130737635adf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:50:51.470471 24402 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca2deb2c9b2d400d9987130737635adf [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:51.472581 24402 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca2deb2c9b2d400d9987130737635adf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca2deb2c9b2d400d9987130737635adf" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33475 } }
I20260619 02:50:51.473099 24402 leader_election.cc:302] T 00000000000000000000000000000000 P ca2deb2c9b2d400d9987130737635adf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca2deb2c9b2d400d9987130737635adf; no voters:
I20260619 02:50:51.473613 24402 leader_election.cc:288] T 00000000000000000000000000000000 P ca2deb2c9b2d400d9987130737635adf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:50:51.473726 24407 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca2deb2c9b2d400d9987130737635adf [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:50:51.474397 24407 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca2deb2c9b2d400d9987130737635adf [term 1 LEADER]: Becoming Leader. State: Replica: ca2deb2c9b2d400d9987130737635adf, State: Running, Role: LEADER
I20260619 02:50:51.474934 24407 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca2deb2c9b2d400d9987130737635adf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca2deb2c9b2d400d9987130737635adf" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33475 } }
I20260619 02:50:51.475579 24402 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca2deb2c9b2d400d9987130737635adf [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:50:51.477557 24409 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca2deb2c9b2d400d9987130737635adf [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca2deb2c9b2d400d9987130737635adf. Latest consensus state: current_term: 1 leader_uuid: "ca2deb2c9b2d400d9987130737635adf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca2deb2c9b2d400d9987130737635adf" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33475 } } }
I20260619 02:50:51.477934 24409 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca2deb2c9b2d400d9987130737635adf [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:51.479144 24408 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca2deb2c9b2d400d9987130737635adf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca2deb2c9b2d400d9987130737635adf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca2deb2c9b2d400d9987130737635adf" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33475 } } }
I20260619 02:50:51.479677 24408 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca2deb2c9b2d400d9987130737635adf [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:51.480212 24416 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:50:51.484899 24416 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:50:51.493535 24416 catalog_manager.cc:1347] Generated new cluster ID: dca436bd8d7d4f6e8b36bda625a3a0dc
I20260619 02:50:51.493647 24416 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:50:51.512570 24416 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:50:51.514408 24416 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:50:51.524992 24416 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca2deb2c9b2d400d9987130737635adf: Generated new TSK 0
I20260619 02:50:51.525856 24416 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:50:51.551244 24405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:51.543030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40936 (local address 127.18.134.62:33475)
0619 02:50:51.543222 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:51.543229 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:51.543743 (+ 514us) server_negotiation.cc:407] Connection header received
0619 02:50:51.544051 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:51.544055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:51.544132 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:51.544263 (+ 131us) server_negotiation.cc:226] Negotiated authn=JWT
0619 02:50:51.544802 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:51.545565 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:51.547532 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:51.548005 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:51.548112 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:51.548731 (+ 619us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 02:50:51.550584 (+ 1853us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35323/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 02:50:51.550947 (+ 363us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40936: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35323/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":61}
W20260619 02:50:51.551390 24405 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40936: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35323/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260619 02:50:51.551448 24431 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:50:51.543020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33475 (local address 127.0.0.1:40936)
0619 02:50:51.543639 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:51.543658 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:51.543806 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:51.544307 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:51.544312 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:51.544326 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0619 02:50:51.544647 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:51.544657 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:51.545701 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:51.545706 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:51.547388 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:51.547403 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:51.548200 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:51.548207 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:51.548401 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:51.548446 (+ 45us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 02:50:51.550795 (+ 2349us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35323/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 02:50:51.551032 (+ 237us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.134.62:33475: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35323/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":302,"threads_started":1}
I20260619 02:50:51.552939 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 24336
I20260619 02:50:51.553915 24397 generic_service.cc:196] Checking for leaks (request via RPC)
E20260619 02:50:51.563169 24432 webserver.cc:533] Webserver: sslize failed:
E20260619 02:50:51.563262 24432 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 02:50:51.642379 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 24336
2026-06-19T02:50:51Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (673 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-19T02:50:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:50:51Z Disabled control of system clock
I20260619 02:50:51.717339 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:37007
--webserver_interface=127.18.134.62
--webserver_port=0
--builtin_ntp_servers=127.18.134.20:36301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:37007
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837410515220-18968-0/testchainca.cert with env {}
W20260619 02:50:52.026860 24437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:52.027166 24437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:52.027271 24437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:52.034622 24437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:50:52.034756 24437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:52.034793 24437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:52.034869 24437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:50:52.034934 24437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 02:50:52.034991 24437 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 02:50:52.035063 24437 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260619 02:50:52.045089 24437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:36301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:37007
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:37007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837410515220-18968-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:52.046381 24437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:52.047669 24437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:52.059077 24445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:52.059252 24443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:52.059674 24442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:52.059891 24437 server_base.cc:1034] running on GCE node
I20260619 02:50:52.060708 24437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:52.062142 24437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:52.063369 24437 hybrid_clock.cc:648] HybridClock initialized: now 1781837452063314 us; error 41 us; skew 500 ppm
I20260619 02:50:52.063735 24437 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:50:52.066022 24437 webserver.cc:460] Webserver started at http://127.18.134.62:38305/ using document root <none> and password file <none>
I20260619 02:50:52.066754 24437 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:52.066860 24437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:52.067139 24437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:52.070057 24437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "97cf3c75c2a84a12a0af016f7eab9468"
format_stamp: "Formatted at 2026-06-19 02:50:52 on dist-test-slave-9mr0"
server_key: "43ba9c0980e0b7c3d0e2fa6c2f278393"
server_key_iv: "87e3a1abbb091bd79b1a79929c528a05"
server_key_version: "encryptionkey@0"
I20260619 02:50:52.070925 24437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "97cf3c75c2a84a12a0af016f7eab9468"
format_stamp: "Formatted at 2026-06-19 02:50:52 on dist-test-slave-9mr0"
server_key: "43ba9c0980e0b7c3d0e2fa6c2f278393"
server_key_iv: "87e3a1abbb091bd79b1a79929c528a05"
server_key_version: "encryptionkey@0"
I20260619 02:50:52.076216 24437 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 02:50:52.079650 24451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:52.080998 24437 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:50:52.081161 24437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "97cf3c75c2a84a12a0af016f7eab9468"
format_stamp: "Formatted at 2026-06-19 02:50:52 on dist-test-slave-9mr0"
server_key: "43ba9c0980e0b7c3d0e2fa6c2f278393"
server_key_iv: "87e3a1abbb091bd79b1a79929c528a05"
server_key_version: "encryptionkey@0"
I20260619 02:50:52.081360 24437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:52.101133 24437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:52.101950 24437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:52.102206 24437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:52.119700 24437 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:37007
I20260619 02:50:52.119745 24502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:37007 every 8 connection(s)
I20260619 02:50:52.120977 24437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:50:52.124652 24503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:52.125160 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 24437
I20260619 02:50:52.125355 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:50:52.126142 18968 external_mini_cluster.cc:1428] Setting key 6990b623aaca9de9fac8d046050da9b9
I20260619 02:50:52.138468 24503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97cf3c75c2a84a12a0af016f7eab9468: Bootstrap starting.
I20260619 02:50:52.140053 24506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:52.128417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51240 (local address 127.18.134.62:37007)
0619 02:50:52.129690 (+ 1273us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:52.129708 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:52.129749 (+ 41us) server_negotiation.cc:407] Connection header received
0619 02:50:52.131353 (+ 1604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:52.131375 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:52.131867 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:52.132201 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:52.132783 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:52.134033 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:52.135780 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:52.136478 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:52.136599 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:52.137772 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:52.137780 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:52.137794 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:50:52.138324 (+ 530us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:52.138400 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:50:52.138421 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:52.138510 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:52.138750 (+ 240us) server_negotiation.cc:299] Negotiation successful
0619 02:50:52.139101 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":731,"thread_start_us":466,"threads_started":1}
I20260619 02:50:52.141666 24503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97cf3c75c2a84a12a0af016f7eab9468: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:52.143006 24503 log.cc:824] T 00000000000000000000000000000000 P 97cf3c75c2a84a12a0af016f7eab9468: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:52.143815 18968 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:50:52.145656 24503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97cf3c75c2a84a12a0af016f7eab9468: No bootstrap required, opened a new log
I20260619 02:50:52.152843 24503 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97cf3c75c2a84a12a0af016f7eab9468 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97cf3c75c2a84a12a0af016f7eab9468" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37007 } }
I20260619 02:50:52.153198 24503 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97cf3c75c2a84a12a0af016f7eab9468 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:52.153281 24503 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97cf3c75c2a84a12a0af016f7eab9468 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97cf3c75c2a84a12a0af016f7eab9468, State: Initialized, Role: FOLLOWER
I20260619 02:50:52.153779 24503 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97cf3c75c2a84a12a0af016f7eab9468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97cf3c75c2a84a12a0af016f7eab9468" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37007 } }
I20260619 02:50:52.153967 24503 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97cf3c75c2a84a12a0af016f7eab9468 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:50:52.154085 24503 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97cf3c75c2a84a12a0af016f7eab9468 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:50:52.154228 24503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97cf3c75c2a84a12a0af016f7eab9468 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:52.156270 24503 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97cf3c75c2a84a12a0af016f7eab9468 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97cf3c75c2a84a12a0af016f7eab9468" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37007 } }
I20260619 02:50:52.156731 24503 leader_election.cc:302] T 00000000000000000000000000000000 P 97cf3c75c2a84a12a0af016f7eab9468 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97cf3c75c2a84a12a0af016f7eab9468; no voters:
I20260619 02:50:52.157277 24503 leader_election.cc:288] T 00000000000000000000000000000000 P 97cf3c75c2a84a12a0af016f7eab9468 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:50:52.157433 24508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97cf3c75c2a84a12a0af016f7eab9468 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:50:52.158032 24508 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97cf3c75c2a84a12a0af016f7eab9468 [term 1 LEADER]: Becoming Leader. State: Replica: 97cf3c75c2a84a12a0af016f7eab9468, State: Running, Role: LEADER
I20260619 02:50:52.158502 24508 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97cf3c75c2a84a12a0af016f7eab9468 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97cf3c75c2a84a12a0af016f7eab9468" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37007 } }
I20260619 02:50:52.159190 24503 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97cf3c75c2a84a12a0af016f7eab9468 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:50:52.161657 24509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97cf3c75c2a84a12a0af016f7eab9468 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97cf3c75c2a84a12a0af016f7eab9468" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97cf3c75c2a84a12a0af016f7eab9468" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37007 } } }
I20260619 02:50:52.161963 24509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97cf3c75c2a84a12a0af016f7eab9468 [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:52.162869 24510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97cf3c75c2a84a12a0af016f7eab9468 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97cf3c75c2a84a12a0af016f7eab9468. Latest consensus state: current_term: 1 leader_uuid: "97cf3c75c2a84a12a0af016f7eab9468" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97cf3c75c2a84a12a0af016f7eab9468" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37007 } } }
I20260619 02:50:52.163272 24510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97cf3c75c2a84a12a0af016f7eab9468 [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:52.163363 24516 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:50:52.168332 24516 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:50:52.176901 24516 catalog_manager.cc:1347] Generated new cluster ID: 82984cf211e14611a72e70f7ccbd3614
I20260619 02:50:52.177024 24516 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:50:52.202775 24516 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:50:52.204042 24516 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:50:52.209496 24516 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97cf3c75c2a84a12a0af016f7eab9468: Generated new TSK 0
I20260619 02:50:52.210316 24516 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:50:52.273743 24506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:52.235225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51242 (local address 127.18.134.62:37007)
0619 02:50:52.235421 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:52.235427 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:52.235727 (+ 300us) server_negotiation.cc:407] Connection header received
0619 02:50:52.235998 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:52.236002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:52.236074 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:52.236206 (+ 132us) server_negotiation.cc:226] Negotiated authn=JWT
0619 02:50:52.236703 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:52.237522 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:52.239582 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:52.240057 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:52.240159 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:52.240744 (+ 585us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 02:50:52.273189 (+ 32445us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 02:50:52.273516 (+ 327us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51242: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":82}
W20260619 02:50:52.273873 24506 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51242: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260619 02:50:52.274206 24532 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:50:52.235013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:37007 (local address 127.0.0.1:51242)
0619 02:50:52.235620 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:52.235639 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:52.235789 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:52.236233 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:52.236239 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:52.236251 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0619 02:50:52.236531 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:52.236541 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:52.237679 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:52.237685 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:52.239416 (+ 1731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:52.239431 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:52.240220 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:52.240227 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:52.240401 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:52.240448 (+ 47us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 02:50:52.273497 (+ 33049us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 02:50:52.273813 (+ 316us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.134.62:37007: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":289,"threads_started":1}
I20260619 02:50:52.275640 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 24437
I20260619 02:50:52.276713 24498 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:52.369922 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 24437
2026-06-19T02:50:52Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (705 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (391 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (398 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:50:53 dist-test-slave-9mr0 krb5kdc[24547](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:50:53 dist-test-slave-9mr0 krb5kdc[24547](info): set up 2 sockets
Jun 19 02:50:53 dist-test-slave-9mr0 krb5kdc[24547](info): commencing operation
krb5kdc: starting...
W20260619 02:50:54.163038 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.972s user 0.009s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:50:54 dist-test-slave-9mr0 krb5kdc[24547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:50:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:50:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:50:54.351997 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:35803
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:42253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:35803
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:54.662026 24560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:54.662333 24560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:54.662446 24560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:54.669595 24560 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:50:54.669709 24560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:54.669754 24560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:54.669826 24560 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:50:54.669888 24560 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:50:54.679924 24560 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:42253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:35803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:35803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:54.681221 24560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:54.682502 24560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:54.693650 24566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:54.693858 24568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:54.693701 24565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:54.694339 24560 server_base.cc:1034] running on GCE node
I20260619 02:50:54.695122 24560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:54.696480 24560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:54.697710 24560 hybrid_clock.cc:648] HybridClock initialized: now 1781837454697658 us; error 37 us; skew 500 ppm
I20260619 02:50:54.698093 24560 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:54 dist-test-slave-9mr0 krb5kdc[24547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837454, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:54.701082 24560 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:50:54.702451 24560 webserver.cc:460] Webserver started at http://127.18.134.62:39823/ using document root <none> and password file <none>
I20260619 02:50:54.703115 24560 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:54.703215 24560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:54.703469 24560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:54.706247 24560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "4149552a5caa4a0cb0fe3eb2accdb9fb"
format_stamp: "Formatted at 2026-06-19 02:50:54 on dist-test-slave-9mr0"
server_key: "53ed06deeaa4b17cdd9ebfb8d8e14bd7"
server_key_iv: "ccb8071669c69cc4791b17eee98a6cb9"
server_key_version: "encryptionkey@0"
I20260619 02:50:54.707003 24560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "4149552a5caa4a0cb0fe3eb2accdb9fb"
format_stamp: "Formatted at 2026-06-19 02:50:54 on dist-test-slave-9mr0"
server_key: "53ed06deeaa4b17cdd9ebfb8d8e14bd7"
server_key_iv: "ccb8071669c69cc4791b17eee98a6cb9"
server_key_version: "encryptionkey@0"
I20260619 02:50:54.711970 24560 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260619 02:50:54.715386 24575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:54.716768 24560 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:50:54.716915 24560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "4149552a5caa4a0cb0fe3eb2accdb9fb"
format_stamp: "Formatted at 2026-06-19 02:50:54 on dist-test-slave-9mr0"
server_key: "53ed06deeaa4b17cdd9ebfb8d8e14bd7"
server_key_iv: "ccb8071669c69cc4791b17eee98a6cb9"
server_key_version: "encryptionkey@0"
I20260619 02:50:54.717132 24560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:54.729713 24560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:54.733880 24560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:54.734123 24560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:54.749595 24560 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:35803
I20260619 02:50:54.749622 24626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:35803 every 8 connection(s)
I20260619 02:50:54.750931 24560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:50:54.751614 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 24560
I20260619 02:50:54.751832 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:50:54.752486 18968 external_mini_cluster.cc:1428] Setting key 79c72cf4c08e9b56f7b49592f2cb61fd
I20260619 02:50:54.755046 24627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 19 02:50:54 dist-test-slave-9mr0 krb5kdc[24547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:54.767285 24627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4149552a5caa4a0cb0fe3eb2accdb9fb: Bootstrap starting.
I20260619 02:50:54.771351 24627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4149552a5caa4a0cb0fe3eb2accdb9fb: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:54.772476 24627 log.cc:824] T 00000000000000000000000000000000 P 4149552a5caa4a0cb0fe3eb2accdb9fb: Log is configured to *not* fsync() on all Append() calls
I20260619 02:50:54.775731 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:54.754278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43156 (local address 127.18.134.62:35803)
0619 02:50:54.754992 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:54.755004 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:54.755045 (+ 41us) server_negotiation.cc:407] Connection header received
0619 02:50:54.756655 (+ 1610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:54.756675 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:54.757055 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:54.757382 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:54.759091 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:54.760494 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:54.762185 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:54.762947 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:54.763078 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:54.767532 (+ 4454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:54.767542 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:54.767567 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:54.767615 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:54.771198 (+ 3583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:54.771939 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:54.771947 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:54.771955 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:54.772151 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:54.772574 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:54.772579 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:54.772582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:54.773259 (+ 677us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:54.773472 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:54.773986 (+ 514us) server_negotiation.cc:299] Negotiation successful
0619 02:50:54.774429 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":322,"threads_started":1}
I20260619 02:50:54.775851 24627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4149552a5caa4a0cb0fe3eb2accdb9fb: No bootstrap required, opened a new log
I20260619 02:50:54.783614 24627 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4149552a5caa4a0cb0fe3eb2accdb9fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4149552a5caa4a0cb0fe3eb2accdb9fb" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35803 } }
I20260619 02:50:54.783972 24627 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4149552a5caa4a0cb0fe3eb2accdb9fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:54.784044 24627 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4149552a5caa4a0cb0fe3eb2accdb9fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4149552a5caa4a0cb0fe3eb2accdb9fb, State: Initialized, Role: FOLLOWER
I20260619 02:50:54.784510 24627 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4149552a5caa4a0cb0fe3eb2accdb9fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4149552a5caa4a0cb0fe3eb2accdb9fb" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35803 } }
I20260619 02:50:54.784696 24627 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4149552a5caa4a0cb0fe3eb2accdb9fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:50:54.784785 24627 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4149552a5caa4a0cb0fe3eb2accdb9fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:50:54.784917 24627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4149552a5caa4a0cb0fe3eb2accdb9fb [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:54.787143 24627 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4149552a5caa4a0cb0fe3eb2accdb9fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4149552a5caa4a0cb0fe3eb2accdb9fb" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35803 } }
I20260619 02:50:54.787588 24627 leader_election.cc:302] T 00000000000000000000000000000000 P 4149552a5caa4a0cb0fe3eb2accdb9fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4149552a5caa4a0cb0fe3eb2accdb9fb; no voters:
I20260619 02:50:54.788141 24627 leader_election.cc:288] T 00000000000000000000000000000000 P 4149552a5caa4a0cb0fe3eb2accdb9fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:50:54.788286 24632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4149552a5caa4a0cb0fe3eb2accdb9fb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:50:54.788880 24632 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4149552a5caa4a0cb0fe3eb2accdb9fb [term 1 LEADER]: Becoming Leader. State: Replica: 4149552a5caa4a0cb0fe3eb2accdb9fb, State: Running, Role: LEADER
I20260619 02:50:54.789398 24632 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4149552a5caa4a0cb0fe3eb2accdb9fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4149552a5caa4a0cb0fe3eb2accdb9fb" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35803 } }
I20260619 02:50:54.790036 24627 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4149552a5caa4a0cb0fe3eb2accdb9fb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:50:54.791917 24633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4149552a5caa4a0cb0fe3eb2accdb9fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4149552a5caa4a0cb0fe3eb2accdb9fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4149552a5caa4a0cb0fe3eb2accdb9fb" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35803 } } }
I20260619 02:50:54.792259 24633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4149552a5caa4a0cb0fe3eb2accdb9fb [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:54.793605 24634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4149552a5caa4a0cb0fe3eb2accdb9fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4149552a5caa4a0cb0fe3eb2accdb9fb. Latest consensus state: current_term: 1 leader_uuid: "4149552a5caa4a0cb0fe3eb2accdb9fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4149552a5caa4a0cb0fe3eb2accdb9fb" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35803 } } }
I20260619 02:50:54.793931 24634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4149552a5caa4a0cb0fe3eb2accdb9fb [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:54.794866 24638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:50:54.800063 24638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:50:54.808578 24638 catalog_manager.cc:1347] Generated new cluster ID: 0e74b63f690b493295afcd8201617b49
I20260619 02:50:54.808714 24638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:50:54.820870 24638 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:50:54.822822 24638 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:50:54.833928 24638 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4149552a5caa4a0cb0fe3eb2accdb9fb: Generated new TSK 0
I20260619 02:50:54.835201 24638 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.134.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
I20260619 02:50:54.881793 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:0
--local_ip_for_outbound_sockets=127.18.134.1
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:35803
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:42253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:55.201726 24653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:55.202018 24653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:55.202152 24653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:55.209481 24653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:55.209589 24653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:55.209733 24653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:50:55.219372 24653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:42253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:35803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:55.220757 24653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:55.222061 24653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:55.234602 24659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:55.234555 24661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:55.234647 24658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:55.244459 24653 server_base.cc:1034] running on GCE node
I20260619 02:50:55.245167 24653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:55.246176 24653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:55.247385 24653 hybrid_clock.cc:648] HybridClock initialized: now 1781837455247332 us; error 44 us; skew 500 ppm
I20260619 02:50:55.247704 24653 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:55 dist-test-slave-9mr0 krb5kdc[24547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:55.250967 24653 init.cc:377] Logged in from keytab as kudu/127.18.134.1@KRBTEST.COM (short username kudu)
I20260619 02:50:55.252475 24653 webserver.cc:460] Webserver started at http://127.18.134.1:46513/ using document root <none> and password file <none>
I20260619 02:50:55.253167 24653 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:55.253270 24653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:55.253544 24653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:55.256398 24653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-0/data/instance:
uuid: "70b047e9063c49e59d54071e4415b36b"
format_stamp: "Formatted at 2026-06-19 02:50:55 on dist-test-slave-9mr0"
server_key: "f4dda8c324621e46105cc82a8239d5aa"
server_key_iv: "316881b480049cb72c6923c99d9554e0"
server_key_version: "encryptionkey@0"
I20260619 02:50:55.257134 24653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance:
uuid: "70b047e9063c49e59d54071e4415b36b"
format_stamp: "Formatted at 2026-06-19 02:50:55 on dist-test-slave-9mr0"
server_key: "f4dda8c324621e46105cc82a8239d5aa"
server_key_iv: "316881b480049cb72c6923c99d9554e0"
server_key_version: "encryptionkey@0"
I20260619 02:50:55.262450 24653 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 02:50:55.265969 24668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:55.267463 24653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:50:55.267620 24653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "70b047e9063c49e59d54071e4415b36b"
format_stamp: "Formatted at 2026-06-19 02:50:55 on dist-test-slave-9mr0"
server_key: "f4dda8c324621e46105cc82a8239d5aa"
server_key_iv: "316881b480049cb72c6923c99d9554e0"
server_key_version: "encryptionkey@0"
I20260619 02:50:55.267818 24653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:55.298435 24653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:55.303004 24653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:55.303308 24653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:55.304302 24653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:50:55.305672 24653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:55.305758 24653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:55.305886 24653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:55.305953 24653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:55.336072 24653 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:38933
I20260619 02:50:55.336117 24780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:38933 every 8 connection(s)
I20260619 02:50:55.337296 24653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:50:55.341284 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 24653
I20260619 02:50:55.341488 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance
I20260619 02:50:55.342222 18968 external_mini_cluster.cc:1428] Setting key def782e90e48346c3a76e200a813ff80
Jun 19 02:50:55 dist-test-slave-9mr0 krb5kdc[24547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:55.357976 24783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:55.339657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:47119 (local address 127.18.134.62:35803)
0619 02:50:55.340066 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:55.340073 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:55.341191 (+ 1118us) server_negotiation.cc:407] Connection header received
0619 02:50:55.343518 (+ 2327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:55.343523 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:55.343597 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:55.343715 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:55.345633 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:55.346310 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:55.348365 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:55.348735 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:55.348825 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:55.352240 (+ 3415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:55.352247 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:55.352252 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:55.352290 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:55.355119 (+ 2829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:55.355763 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:55.355767 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:55.355770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:55.355909 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:55.356266 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:55.356270 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:55.356273 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:55.356591 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:55.356727 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:55.357378 (+ 651us) server_negotiation.cc:299] Negotiation successful
0619 02:50:55.357586 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":187,"threads_started":1}
I20260619 02:50:55.358912 24784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:55.340267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35803 (local address 127.18.134.1:47119)
0619 02:50:55.341006 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:55.341059 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:55.343242 (+ 2183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:55.343938 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:55.343951 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:55.344647 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:55.345452 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:55.345478 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:55.346466 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:55.346470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:55.348244 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:55.348257 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:55.348953 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:55.348959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:55.349216 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:55.349881 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:55.349919 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:55.352037 (+ 2118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:55.355242 (+ 3205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:55.355248 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:55.355262 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:55.355649 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:55.355992 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:55.355995 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:55.355997 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:55.356167 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:55.356690 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:55.356696 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:55.357031 (+ 335us) client_negotiation.cc:769] Sending connection context
0619 02:50:55.357243 (+ 212us) client_negotiation.cc:240] Negotiation successful
0619 02:50:55.357623 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":270,"threads_started":1}
I20260619 02:50:55.360663 24781 heartbeater.cc:347] Connected to a master server at 127.18.134.62:35803
I20260619 02:50:55.361037 24781 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.18.134.2@KRBTEST.COM; defaulting to no policy
I20260619 02:50:55.362035 24781 heartbeater.cc:511] Master 127.18.134.62:35803 requested a full tablet report, sending...
I20260619 02:50:55.364409 24592 ts_manager.cc:194] Registered new tserver with Master: 70b047e9063c49e59d54071e4415b36b (127.18.134.1:38933)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.2@KRBTEST.COM" created.
I20260619 02:50:55.365931 24592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.1@KRBTEST.COM'} at 127.18.134.1:47119
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
I20260619 02:50:55.387183 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:0
--local_ip_for_outbound_sockets=127.18.134.2
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:35803
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:42253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:55.700414 24787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:55.700716 24787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:55.700845 24787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:55.708127 24787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:55.708252 24787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:55.708406 24787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:50:55.718078 24787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:42253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:35803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:55.719568 24787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:55.720871 24787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:55.734485 24792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:55.734509 24793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:55.734737 24795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:55.735342 24787 server_base.cc:1034] running on GCE node
I20260619 02:50:55.735951 24787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:55.736922 24787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:55.738119 24787 hybrid_clock.cc:648] HybridClock initialized: now 1781837455738050 us; error 57 us; skew 500 ppm
I20260619 02:50:55.738482 24787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:55 dist-test-slave-9mr0 krb5kdc[24547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:55.741557 24787 init.cc:377] Logged in from keytab as kudu/127.18.134.2@KRBTEST.COM (short username kudu)
I20260619 02:50:55.743098 24787 webserver.cc:460] Webserver started at http://127.18.134.2:37101/ using document root <none> and password file <none>
I20260619 02:50:55.743769 24787 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:55.743882 24787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:55.744186 24787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:55.747105 24787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-1/data/instance:
uuid: "eb0af632f6e04714af65f89b6e110eff"
format_stamp: "Formatted at 2026-06-19 02:50:55 on dist-test-slave-9mr0"
server_key: "04729b830ddba1aff32f7c6053765834"
server_key_iv: "d7d6b3810bc0dc021b7951567aca5705"
server_key_version: "encryptionkey@0"
I20260619 02:50:55.747905 24787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance:
uuid: "eb0af632f6e04714af65f89b6e110eff"
format_stamp: "Formatted at 2026-06-19 02:50:55 on dist-test-slave-9mr0"
server_key: "04729b830ddba1aff32f7c6053765834"
server_key_iv: "d7d6b3810bc0dc021b7951567aca5705"
server_key_version: "encryptionkey@0"
I20260619 02:50:55.753593 24787 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 02:50:55.757258 24802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:55.758709 24787 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260619 02:50:55.758867 24787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "eb0af632f6e04714af65f89b6e110eff"
format_stamp: "Formatted at 2026-06-19 02:50:55 on dist-test-slave-9mr0"
server_key: "04729b830ddba1aff32f7c6053765834"
server_key_iv: "d7d6b3810bc0dc021b7951567aca5705"
server_key_version: "encryptionkey@0"
I20260619 02:50:55.759058 24787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:55.793696 24787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:55.798120 24787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:55.798417 24787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:55.799428 24787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:50:55.800930 24787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:55.801029 24787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:55.801158 24787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:55.801208 24787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:55.831350 24787 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:33485
I20260619 02:50:55.831400 24914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:33485 every 8 connection(s)
I20260619 02:50:55.832616 24787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:50:55.836187 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 24787
I20260619 02:50:55.836361 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance
I20260619 02:50:55.836906 18968 external_mini_cluster.cc:1428] Setting key 2e58b1a927f18b85d905564a795c721e
Jun 19 02:50:55 dist-test-slave-9mr0 krb5kdc[24547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837455, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:55.854272 24783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:55.835392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:40095 (local address 127.18.134.62:35803)
0619 02:50:55.835552 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:55.835557 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:55.836979 (+ 1422us) server_negotiation.cc:407] Connection header received
0619 02:50:55.839220 (+ 2241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:55.839225 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:55.839294 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:55.839377 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:55.841310 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:55.842020 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:55.844075 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:55.844496 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:55.844588 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:55.848268 (+ 3680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:55.848275 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:55.848279 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:55.848315 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:55.851191 (+ 2876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:55.851880 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:55.851885 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:55.851889 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:55.852043 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:55.852486 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:55.852490 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:55.852493 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:55.852814 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:55.852970 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:55.853702 (+ 732us) server_negotiation.cc:299] Negotiation successful
0619 02:50:55.853924 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 02:50:55.855127 24917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:55.835975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35803 (local address 127.18.134.2:40095)
0619 02:50:55.836800 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:55.836852 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:55.838906 (+ 2054us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:55.839660 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:55.839673 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:55.840349 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:55.841128 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:55.841151 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:55.842159 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:55.842164 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:55.843950 (+ 1786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:55.843963 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:55.844699 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:55.844706 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:55.844976 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:55.845723 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:55.845750 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:55.848030 (+ 2280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:55.851332 (+ 3302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:55.851338 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:55.851351 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:55.851754 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:55.852171 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:55.852174 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:55.852177 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:55.852378 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:55.852961 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:55.852977 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:55.853336 (+ 359us) client_negotiation.cc:769] Sending connection context
0619 02:50:55.853557 (+ 221us) client_negotiation.cc:240] Negotiation successful
0619 02:50:55.853962 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":356,"threads_started":1}
I20260619 02:50:55.856492 24915 heartbeater.cc:347] Connected to a master server at 127.18.134.62:35803
WARNING: no policy specified for kudu/127.18.134.3@KRBTEST.COM; defaulting to no policy
I20260619 02:50:55.856832 24915 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:55.857605 24915 heartbeater.cc:511] Master 127.18.134.62:35803 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.3@KRBTEST.COM" created.
I20260619 02:50:55.859171 24592 ts_manager.cc:194] Registered new tserver with Master: eb0af632f6e04714af65f89b6e110eff (127.18.134.2:33485)
I20260619 02:50:55.860078 24592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.2@KRBTEST.COM'} at 127.18.134.2:40095
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
I20260619 02:50:55.880028 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:0
--local_ip_for_outbound_sockets=127.18.134.3
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:35803
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:42253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:56.198346 24920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:56.198621 24920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:56.198748 24920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:56.205884 24920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:56.205993 24920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:56.206110 24920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:50:56.216356 24920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:42253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:35803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:56.217588 24920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:56.218884 24920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:56.231446 24926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:56.231503 24925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:56.231480 24928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:56.232223 24920 server_base.cc:1034] running on GCE node
I20260619 02:50:56.232807 24920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:56.233732 24920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:56.234930 24920 hybrid_clock.cc:648] HybridClock initialized: now 1781837456234873 us; error 50 us; skew 500 ppm
I20260619 02:50:56.235224 24920 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:56 dist-test-slave-9mr0 krb5kdc[24547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837456, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:56.238132 24920 init.cc:377] Logged in from keytab as kudu/127.18.134.3@KRBTEST.COM (short username kudu)
I20260619 02:50:56.239574 24920 webserver.cc:460] Webserver started at http://127.18.134.3:34825/ using document root <none> and password file <none>
I20260619 02:50:56.240221 24920 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:56.240324 24920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:56.240584 24920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:56.243417 24920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-2/data/instance:
uuid: "5af824af50534de18381882b7347c849"
format_stamp: "Formatted at 2026-06-19 02:50:56 on dist-test-slave-9mr0"
server_key: "43ec3738e297e5438d597e17c44d6803"
server_key_iv: "372f1a9c3a655e3493ff9522f7379af9"
server_key_version: "encryptionkey@0"
I20260619 02:50:56.244122 24920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance:
uuid: "5af824af50534de18381882b7347c849"
format_stamp: "Formatted at 2026-06-19 02:50:56 on dist-test-slave-9mr0"
server_key: "43ec3738e297e5438d597e17c44d6803"
server_key_iv: "372f1a9c3a655e3493ff9522f7379af9"
server_key_version: "encryptionkey@0"
I20260619 02:50:56.249137 24920 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:50:56.252547 24935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:56.253813 24920 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:50:56.253959 24920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "5af824af50534de18381882b7347c849"
format_stamp: "Formatted at 2026-06-19 02:50:56 on dist-test-slave-9mr0"
server_key: "43ec3738e297e5438d597e17c44d6803"
server_key_iv: "372f1a9c3a655e3493ff9522f7379af9"
server_key_version: "encryptionkey@0"
I20260619 02:50:56.254133 24920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:56.272734 24920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:56.277066 24920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:56.277341 24920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:56.278267 24920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:50:56.279676 24920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:56.279757 24920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:56.279881 24920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:56.279933 24920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:56.310070 24920 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:33069
I20260619 02:50:56.310112 25047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:33069 every 8 connection(s)
I20260619 02:50:56.311333 24920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:50:56.318522 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 24920
I20260619 02:50:56.318724 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance
I20260619 02:50:56.319214 18968 external_mini_cluster.cc:1428] Setting key 69c61d12c8bdcf69a773543dee674229
Jun 19 02:50:56 dist-test-slave-9mr0 krb5kdc[24547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837456, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:50:56.330032 24783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:56.313680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:48075 (local address 127.18.134.62:35803)
0619 02:50:56.313840 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:56.313846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:56.314986 (+ 1140us) server_negotiation.cc:407] Connection header received
0619 02:50:56.316640 (+ 1654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:56.316645 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:56.316734 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:56.316817 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:56.318223 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:56.319300 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:56.321076 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:56.321479 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:56.321582 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:56.324725 (+ 3143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:56.324730 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:56.324733 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:56.324764 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:56.327359 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:56.328077 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:56.328081 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:56.328083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:56.328225 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:56.328619 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:56.328622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:56.328624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:56.328897 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:56.329008 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:56.329610 (+ 602us) server_negotiation.cc:299] Negotiation successful
0619 02:50:56.329762 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 02:50:56.331008 25050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:56.314118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35803 (local address 127.18.134.3:48075)
0619 02:50:56.314828 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:56.314872 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:56.316410 (+ 1538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:56.317004 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:56.317012 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:56.317504 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:56.318053 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:56.318069 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:56.319434 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:56.319438 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:56.320940 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:56.320953 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:56.321681 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:56.321688 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:56.321928 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:56.322491 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:56.322515 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:56.324536 (+ 2021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:56.327486 (+ 2950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:56.327493 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:56.327506 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:56.327952 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:56.328334 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:56.328337 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:56.328340 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:56.328521 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:56.329006 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:56.329011 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:56.329325 (+ 314us) client_negotiation.cc:769] Sending connection context
0619 02:50:56.329551 (+ 226us) client_negotiation.cc:240] Negotiation successful
0619 02:50:56.329932 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":266,"threads_started":1}
I20260619 02:50:56.332278 25048 heartbeater.cc:347] Connected to a master server at 127.18.134.62:35803
I20260619 02:50:56.332573 25048 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:56.333321 25048 heartbeater.cc:511] Master 127.18.134.62:35803 requested a full tablet report, sending...
I20260619 02:50:56.334677 24592 ts_manager.cc:194] Registered new tserver with Master: 5af824af50534de18381882b7347c849 (127.18.134.3:33069)
I20260619 02:50:56.335438 24592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.3@KRBTEST.COM'} at 127.18.134.3:48075
I20260619 02:50:56.346033 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/test-admin.keytab.
I20260619 02:50:56.368966 24781 heartbeater.cc:503] Master 127.18.134.62:35803 was elected leader, sending a full tablet report...
Jun 19 02:50:56 dist-test-slave-9mr0 krb5kdc[24547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:56.371876 18968 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260619 02:50:56.376520 25055 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:50:56.374256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34684 (local address 127.18.134.1:38933)
0619 02:50:56.374856 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:56.374863 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:56.374886 (+ 23us) server_negotiation.cc:407] Connection header received
0619 02:50:56.374937 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:56.374942 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:56.375150 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:56.375269 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:56.376113 (+ 844us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34684: BlockingRecv error: recv got EOF from 127.0.0.1:34684 (error 108)
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":299,"threads_started":1}
W20260619 02:50:56.376566 18968 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/krb5cc' of type FILE
Jun 19 02:50:56 dist-test-slave-9mr0 krb5kdc[24547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:56.378057 18968 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 02:50:56 dist-test-slave-9mr0 krb5kdc[24547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:56.379359 18968 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 02:50:56 dist-test-slave-9mr0 krb5kdc[24547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.1@KRBTEST.COM
I20260619 02:50:56.393738 25055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:56.380963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34692 (local address 127.18.134.1:38933)
0619 02:50:56.381135 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:56.381141 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:56.381296 (+ 155us) server_negotiation.cc:407] Connection header received
0619 02:50:56.381469 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:56.381473 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:56.381528 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:56.381598 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:56.382515 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:56.383500 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:56.384663 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:56.385206 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:56.385318 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:56.388549 (+ 3231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:56.388560 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:56.388574 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:56.388605 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:56.391189 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:56.391792 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:56.391799 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:56.391801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:56.391917 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:56.392247 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:56.392251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:56.392253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:56.392601 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:56.392864 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:56.393209 (+ 345us) server_negotiation.cc:299] Negotiation successful
0619 02:50:56.393464 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260619 02:50:56.398862 25055 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:50:56.397453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34708 (local address 127.18.134.1:38933)
0619 02:50:56.397647 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:56.397652 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:56.397873 (+ 221us) server_negotiation.cc:407] Connection header received
0619 02:50:56.397966 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:56.397969 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:56.398021 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:56.398094 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:56.398604 (+ 510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34708: BlockingRecv error: recv got EOF from 127.0.0.1:34708 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260619 02:50:56.399307 18968 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/krb5cc)
Jun 19 02:50:56 dist-test-slave-9mr0 krb5kdc[24547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:56.400660 18968 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 02:50:56 dist-test-slave-9mr0 krb5kdc[24547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:56.401813 18968 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 02:50:56 dist-test-slave-9mr0 krb5kdc[24547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.1@KRBTEST.COM
I20260619 02:50:56.414664 25055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:56.403004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34714 (local address 127.18.134.1:38933)
0619 02:50:56.403175 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:56.403180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:56.403337 (+ 157us) server_negotiation.cc:407] Connection header received
0619 02:50:56.403445 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:56.403448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:56.403499 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:56.403569 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:56.404403 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:56.405133 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:56.406377 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:56.406817 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:56.406933 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:56.410288 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:56.410293 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:56.410296 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:56.410327 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:56.412565 (+ 2238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:56.413126 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:56.413130 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:56.413132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:56.413253 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:56.413534 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:56.413537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:56.413539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:56.413825 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:56.414008 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:56.414206 (+ 198us) server_negotiation.cc:299] Negotiation successful
0619 02:50:56.414357 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260619 02:50:56.418383 25055 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:50:56.416902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34716 (local address 127.18.134.1:38933)
0619 02:50:56.417048 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:56.417054 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:56.417301 (+ 247us) server_negotiation.cc:407] Connection header received
0619 02:50:56.417440 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:56.417444 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:56.417504 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:56.417577 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:56.418199 (+ 622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34716: BlockingRecv error: recv got EOF from 127.0.0.1:34716 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260619 02:50:56.418579 18968 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781837410515220-18968-0/krb5kdc/krb5cc' of type FILE
Jun 19 02:50:56 dist-test-slave-9mr0 krb5kdc[24547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:56.420058 18968 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 02:50:56 dist-test-slave-9mr0 krb5kdc[24547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:56.421252 18968 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 02:50:56 dist-test-slave-9mr0 krb5kdc[24547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.1@KRBTEST.COM
I20260619 02:50:56.434255 25055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:56.422459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34718 (local address 127.18.134.1:38933)
0619 02:50:56.422668 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:56.422673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:56.422895 (+ 222us) server_negotiation.cc:407] Connection header received
0619 02:50:56.423002 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:56.423005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:56.423057 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:56.423129 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:56.423982 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:56.424742 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:56.425935 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:56.426412 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:56.426506 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:56.429763 (+ 3257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:56.429768 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:56.429771 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:56.429801 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:56.432126 (+ 2325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:56.432708 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:56.432711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:56.432714 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:56.432845 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:56.433159 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:56.433162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:56.433164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:56.433417 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:56.433632 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:56.433784 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 02:50:56.433937 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 02:50:56.435668 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 24653
I20260619 02:50:56.446656 25055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:56.436292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34732 (local address 127.18.134.1:38933)
0619 02:50:56.436483 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:56.436489 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:56.436789 (+ 300us) server_negotiation.cc:407] Connection header received
0619 02:50:56.436875 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:56.436878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:56.436932 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:56.437014 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:56.437820 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:56.438479 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:56.439672 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:56.440127 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:56.440224 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:56.442092 (+ 1868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:56.442096 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:56.442099 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:56.442131 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:56.444408 (+ 2277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:56.444993 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:56.444997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:56.444999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:56.445115 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:56.445402 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:56.445405 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:56.445407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:56.445636 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:56.445831 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:56.446077 (+ 246us) server_negotiation.cc:299] Negotiation successful
0619 02:50:56.446284 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 02:50:56.447314 24776 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:56.527879 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 24653
I20260619 02:50:56.540115 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 24787
Jun 19 02:50:56 dist-test-slave-9mr0 krb5kdc[24547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.2@KRBTEST.COM
I20260619 02:50:56.554136 25068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:56.540914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56688 (local address 127.18.134.2:33485)
0619 02:50:56.541385 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:56.541392 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:56.541418 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:50:56.541474 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:56.541481 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:56.541687 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:56.541817 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:56.542804 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:56.543785 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:56.544981 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:56.545525 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:56.545627 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:56.549073 (+ 3446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:56.549084 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:56.549098 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:56.549131 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:56.551879 (+ 2748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:56.552491 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:56.552497 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:56.552500 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:56.552613 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:56.552895 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:56.552898 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:56.552900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:56.553217 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:56.553436 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:56.553643 (+ 207us) server_negotiation.cc:299] Negotiation successful
0619 02:50:56.553839 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":233,"threads_started":1}
I20260619 02:50:56.555399 24910 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:56.634603 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 24787
I20260619 02:50:56.646433 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 24920
Jun 19 02:50:56 dist-test-slave-9mr0 krb5kdc[24547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.3@KRBTEST.COM
I20260619 02:50:56.661033 25070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:56.647299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40878 (local address 127.18.134.3:33069)
0619 02:50:56.647883 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:56.647890 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:56.647919 (+ 29us) server_negotiation.cc:407] Connection header received
0619 02:50:56.647975 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:56.647981 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:56.648185 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:56.648336 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:56.649460 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:56.650492 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:56.651807 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:56.652371 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:56.652471 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:56.656061 (+ 3590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:56.656072 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:56.656087 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:56.656120 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:56.658849 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:56.659417 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:56.659423 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:56.659425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:56.659544 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:56.659804 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:56.659806 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:56.659808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:56.660107 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:56.660313 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:56.660526 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 02:50:56.660729 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":271,"threads_started":1}
I20260619 02:50:56.662216 25043 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:56.743572 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 24920
I20260619 02:50:56.755295 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 24560
I20260619 02:50:56.756373 24622 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:50:56.854017 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 24560
2026-06-19T02:50:56Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3699 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:50:56 dist-test-slave-9mr0 krb5kdc[25075](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:50:56 dist-test-slave-9mr0 krb5kdc[25075](info): set up 2 sockets
Jun 19 02:50:56 dist-test-slave-9mr0 krb5kdc[25075](info): commencing operation
krb5kdc: starting...
W20260619 02:50:57.851035 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.961s user 0.001s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:50:57 dist-test-slave-9mr0 krb5kdc[25075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:50:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:50:57Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.62.keytab.
Entry for principal oryx/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.62.keytab.
I20260619 02:50:58.037628 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:33075
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.62.keytab
--principal=oryx/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:35501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:33075
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:58.342824 25088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:58.343113 25088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:58.343247 25088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:58.350303 25088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:50:58.350414 25088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:58.350490 25088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:58.350571 25088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:50:58.350652 25088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:50:58.360200 25088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:35501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:33075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.62.keytab
--principal=oryx/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:33075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:58.361461 25088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:58.362722 25088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:58.373764 25093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:58.373904 25094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:58.373736 25096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:58.374672 25088 server_base.cc:1034] running on GCE node
I20260619 02:50:58.375432 25088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:58.376698 25088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:58.377895 25088 hybrid_clock.cc:648] HybridClock initialized: now 1781837458377857 us; error 30 us; skew 500 ppm
I20260619 02:50:58.378221 25088 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:58 dist-test-slave-9mr0 krb5kdc[25075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837458, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:58.385376 25088 init.cc:377] Logged in from keytab as oryx/127.18.134.62@KRBTEST.COM (short username oryx)
I20260619 02:50:58.386955 25088 webserver.cc:460] Webserver started at http://127.18.134.62:33331/ using document root <none> and password file <none>
I20260619 02:50:58.387619 25088 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:58.387732 25088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:58.388018 25088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:58.390915 25088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "0094edcfb4284971a6d45b1ec4e48a30"
format_stamp: "Formatted at 2026-06-19 02:50:58 on dist-test-slave-9mr0"
server_key: "dc0d874256d47e4c09596bbd6b52a2da"
server_key_iv: "5d6a2904eba64f1283a0286f68fb28dd"
server_key_version: "encryptionkey@0"
I20260619 02:50:58.391611 25088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "0094edcfb4284971a6d45b1ec4e48a30"
format_stamp: "Formatted at 2026-06-19 02:50:58 on dist-test-slave-9mr0"
server_key: "dc0d874256d47e4c09596bbd6b52a2da"
server_key_iv: "5d6a2904eba64f1283a0286f68fb28dd"
server_key_version: "encryptionkey@0"
I20260619 02:50:58.397217 25088 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260619 02:50:58.400462 25103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:58.401791 25088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:50:58.401978 25088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "0094edcfb4284971a6d45b1ec4e48a30"
format_stamp: "Formatted at 2026-06-19 02:50:58 on dist-test-slave-9mr0"
server_key: "dc0d874256d47e4c09596bbd6b52a2da"
server_key_iv: "5d6a2904eba64f1283a0286f68fb28dd"
server_key_version: "encryptionkey@0"
I20260619 02:50:58.402146 25088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:58.427215 25088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:58.431447 25088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:58.431743 25088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:58.448200 25088 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:33075
I20260619 02:50:58.448251 25154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:33075 every 8 connection(s)
I20260619 02:50:58.449473 25088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:50:58.453053 25155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:50:58.455402 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 25088
I20260619 02:50:58.455561 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:50:58.456203 18968 external_mini_cluster.cc:1428] Setting key f627ad687cfe546623734197417888f0
I20260619 02:50:58.464099 25155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30: Bootstrap starting.
I20260619 02:50:58.467568 25155 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30: Neither blocks nor log segments found. Creating new log.
I20260619 02:50:58.468780 25155 log.cc:824] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30: Log is configured to *not* fsync() on all Append() calls
Jun 19 02:50:58 dist-test-slave-9mr0 krb5kdc[25075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM
I20260619 02:50:58.471866 25155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30: No bootstrap required, opened a new log
I20260619 02:50:58.477515 25158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:58.458026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38512 (local address 127.18.134.62:33075)
0619 02:50:58.458705 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:58.458717 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:58.458755 (+ 38us) server_negotiation.cc:407] Connection header received
0619 02:50:58.460319 (+ 1564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:58.460339 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:58.460712 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:58.461029 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:58.462680 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:58.463936 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:58.465576 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:58.466219 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:58.466341 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:58.470907 (+ 4566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:58.470915 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:58.470931 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:58.470967 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:58.473822 (+ 2855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:58.474554 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:58.474560 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:58.474566 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:58.474747 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:58.475094 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:58.475097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:58.475099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:58.475572 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:58.475783 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:58.476181 (+ 398us) server_negotiation.cc:299] Negotiation successful
0619 02:50:58.476505 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":285,"threads_started":1}
I20260619 02:50:58.480060 25155 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } }
I20260619 02:50:58.480391 25155 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:50:58.480476 25155 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0094edcfb4284971a6d45b1ec4e48a30, State: Initialized, Role: FOLLOWER
I20260619 02:50:58.481032 25155 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } }
I20260619 02:50:58.481235 25155 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:50:58.481348 25155 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:50:58.481485 25155 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:50:58.483541 25155 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } }
I20260619 02:50:58.484053 25155 leader_election.cc:302] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0094edcfb4284971a6d45b1ec4e48a30; no voters:
I20260619 02:50:58.484673 25155 leader_election.cc:288] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:50:58.484748 25160 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:50:58.485343 25160 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [term 1 LEADER]: Becoming Leader. State: Replica: 0094edcfb4284971a6d45b1ec4e48a30, State: Running, Role: LEADER
I20260619 02:50:58.485822 25160 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } }
I20260619 02:50:58.486488 25155 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:50:58.488399 25161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0094edcfb4284971a6d45b1ec4e48a30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } }
I20260619 02:50:58.488745 25161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:58.489403 25162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0094edcfb4284971a6d45b1ec4e48a30. Latest consensus state: current_term: 1 leader_uuid: "0094edcfb4284971a6d45b1ec4e48a30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } }
I20260619 02:50:58.489724 25162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [sys.catalog]: This master's current role is: LEADER
I20260619 02:50:58.491027 25167 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:50:58.495831 25167 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:50:58.503248 25167 catalog_manager.cc:1347] Generated new cluster ID: 9cbe8949e88b4785a8e697bf8dceef26
I20260619 02:50:58.503361 25167 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:50:58.521317 25167 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:50:58.523103 25167 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:50:58.531980 25167 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30: Generated new TSK 0
I20260619 02:50:58.532992 25167 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.134.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.134.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.134.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.1.keytab.
Entry for principal oryx/127.18.134.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.1.keytab.
I20260619 02:50:58.586181 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:0
--local_ip_for_outbound_sockets=127.18.134.1
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:33075
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.1.keytab
--principal=oryx/127.18.134.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:35501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:58.893204 25181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:58.893468 25181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:58.893558 25181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:58.901036 25181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:58.901144 25181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:58.901257 25181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:50:58.910892 25181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:35501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.1.keytab
--principal=oryx/127.18.134.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:33075
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:58.912122 25181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:58.913424 25181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:58.925770 25187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:58.925743 25189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:58.925709 25186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:58.926947 25181 server_base.cc:1034] running on GCE node
I20260619 02:50:58.927511 25181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:58.928465 25181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:58.929664 25181 hybrid_clock.cc:648] HybridClock initialized: now 1781837458929615 us; error 39 us; skew 500 ppm
I20260619 02:50:58.929975 25181 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:58 dist-test-slave-9mr0 krb5kdc[25075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837458, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:58.933009 25181 init.cc:377] Logged in from keytab as oryx/127.18.134.1@KRBTEST.COM (short username oryx)
I20260619 02:50:58.934386 25181 webserver.cc:460] Webserver started at http://127.18.134.1:35633/ using document root <none> and password file <none>
I20260619 02:50:58.935104 25181 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:58.935218 25181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:58.935492 25181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:58.938277 25181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/data/instance:
uuid: "10455eb59a7040a98cb6dc8988bc24b8"
format_stamp: "Formatted at 2026-06-19 02:50:58 on dist-test-slave-9mr0"
server_key: "bac4f025e19ee134097c8a184a94f407"
server_key_iv: "46ad1e3f531f44b366eec545b6c2db3b"
server_key_version: "encryptionkey@0"
I20260619 02:50:58.939031 25181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance:
uuid: "10455eb59a7040a98cb6dc8988bc24b8"
format_stamp: "Formatted at 2026-06-19 02:50:58 on dist-test-slave-9mr0"
server_key: "bac4f025e19ee134097c8a184a94f407"
server_key_iv: "46ad1e3f531f44b366eec545b6c2db3b"
server_key_version: "encryptionkey@0"
I20260619 02:50:58.943874 25181 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 02:50:58.946976 25196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:58.948194 25181 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260619 02:50:58.948339 25181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "10455eb59a7040a98cb6dc8988bc24b8"
format_stamp: "Formatted at 2026-06-19 02:50:58 on dist-test-slave-9mr0"
server_key: "bac4f025e19ee134097c8a184a94f407"
server_key_iv: "46ad1e3f531f44b366eec545b6c2db3b"
server_key_version: "encryptionkey@0"
I20260619 02:50:58.948501 25181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:58.965201 25181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:58.969156 25181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:58.969419 25181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:58.970378 25181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:50:58.971796 25181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:58.971879 25181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:58.971997 25181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:58.972048 25181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:59.002005 25181 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:45679
I20260619 02:50:59.002040 25308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:45679 every 8 connection(s)
I20260619 02:50:59.003276 25181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:50:59.004261 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 25181
I20260619 02:50:59.004477 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance
I20260619 02:50:59.005230 18968 external_mini_cluster.cc:1428] Setting key 90eeda0fcbb4cb1e2356a03260bede2d
Jun 19 02:50:59 dist-test-slave-9mr0 krb5kdc[25075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837458, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM
WARNING: no policy specified for oryx/127.18.134.2@KRBTEST.COM; defaulting to no policy
I20260619 02:50:59.022994 25311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:59.005956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:59755 (local address 127.18.134.62:33075)
0619 02:50:59.006343 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:59.006348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:59.007520 (+ 1172us) server_negotiation.cc:407] Connection header received
0619 02:50:59.009302 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:59.009307 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:59.009388 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:59.009487 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:59.010991 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:59.011937 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:59.013457 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:59.013846 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:59.013954 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:59.017355 (+ 3401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:59.017360 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:59.017363 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:59.017400 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:59.019795 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:59.020630 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:59.020634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:59.020636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:59.020741 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:59.021171 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:59.021174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:59.021176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:59.021436 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:59.021537 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:59.022456 (+ 919us) server_negotiation.cc:299] Negotiation successful
0619 02:50:59.022673 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":183,"threads_started":1}
I20260619 02:50:59.023978 25312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:59.006571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.18.134.1:59755)
0619 02:50:59.007372 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:59.007418 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:59.009048 (+ 1630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:59.009663 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:59.009671 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:59.010160 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:59.010819 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:59.010839 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:59.012084 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:59.012088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:59.013299 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:59.013310 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:59.014074 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:59.014081 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:59.014288 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:59.014951 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:59.014972 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:59.017156 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:59.019939 (+ 2783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:59.019949 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:59.019967 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:59.020527 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:59.020833 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:59.020838 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:59.020842 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:59.021077 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:59.021551 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:59.021560 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:59.022005 (+ 445us) client_negotiation.cc:769] Sending connection context
0619 02:50:59.022314 (+ 309us) client_negotiation.cc:240] Negotiation successful
0619 02:50:59.022883 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":302,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.134.2@KRBTEST.COM" created.
I20260619 02:50:59.025674 25309 heartbeater.cc:347] Connected to a master server at 127.18.134.62:33075
I20260619 02:50:59.025988 25309 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:59.026775 25309 heartbeater.cc:511] Master 127.18.134.62:33075 requested a full tablet report, sending...
I20260619 02:50:59.028824 25120 ts_manager.cc:194] Registered new tserver with Master: 10455eb59a7040a98cb6dc8988bc24b8 (127.18.134.1:45679)
I20260619 02:50:59.030262 25120 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.134.1@KRBTEST.COM'} at 127.18.134.1:59755
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.134.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.2.keytab.
Entry for principal oryx/127.18.134.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.2.keytab.
I20260619 02:50:59.048956 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:0
--local_ip_for_outbound_sockets=127.18.134.2
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:33075
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.2.keytab
--principal=oryx/127.18.134.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:35501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:59.354832 25315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:59.355113 25315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:59.355242 25315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:59.362594 25315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:59.362735 25315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:59.362913 25315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:50:59.372349 25315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:35501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.2.keytab
--principal=oryx/127.18.134.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:33075
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:59.373580 25315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:59.374898 25315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:59.387276 25321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:59.387313 25323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:59.387334 25320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:59.388139 25315 server_base.cc:1034] running on GCE node
I20260619 02:50:59.388782 25315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:59.389926 25315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:59.391170 25315 hybrid_clock.cc:648] HybridClock initialized: now 1781837459391084 us; error 79 us; skew 500 ppm
I20260619 02:50:59.391532 25315 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:59 dist-test-slave-9mr0 krb5kdc[25075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837459, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:59.394531 25315 init.cc:377] Logged in from keytab as oryx/127.18.134.2@KRBTEST.COM (short username oryx)
I20260619 02:50:59.395887 25315 webserver.cc:460] Webserver started at http://127.18.134.2:38569/ using document root <none> and password file <none>
I20260619 02:50:59.396538 25315 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:59.396651 25315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:59.396932 25315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:59.399837 25315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/data/instance:
uuid: "f46cfab95e6e44c0be837c69674063c9"
format_stamp: "Formatted at 2026-06-19 02:50:59 on dist-test-slave-9mr0"
server_key: "598037b8ebb881b36a0a508f9072ec38"
server_key_iv: "0a42beded8afe2e32fedb28aa286b75f"
server_key_version: "encryptionkey@0"
I20260619 02:50:59.400559 25315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance:
uuid: "f46cfab95e6e44c0be837c69674063c9"
format_stamp: "Formatted at 2026-06-19 02:50:59 on dist-test-slave-9mr0"
server_key: "598037b8ebb881b36a0a508f9072ec38"
server_key_iv: "0a42beded8afe2e32fedb28aa286b75f"
server_key_version: "encryptionkey@0"
I20260619 02:50:59.405542 25315 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:50:59.408965 25330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:59.410276 25315 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 02:50:59.410418 25315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "f46cfab95e6e44c0be837c69674063c9"
format_stamp: "Formatted at 2026-06-19 02:50:59 on dist-test-slave-9mr0"
server_key: "598037b8ebb881b36a0a508f9072ec38"
server_key_iv: "0a42beded8afe2e32fedb28aa286b75f"
server_key_version: "encryptionkey@0"
I20260619 02:50:59.410600 25315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:59.439299 25315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:59.443351 25315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:59.443619 25315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:59.444521 25315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:50:59.445815 25315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:59.445890 25315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:59.446002 25315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:59.446060 25315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:59.476096 25315 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:45195
I20260619 02:50:59.476164 25442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:45195 every 8 connection(s)
I20260619 02:50:59.477303 25315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:50:59.478421 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 25315
I20260619 02:50:59.478605 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance
I20260619 02:50:59.479137 18968 external_mini_cluster.cc:1428] Setting key 73aa1d92c192ab9940207aa5ba58c612
Jun 19 02:50:59 dist-test-slave-9mr0 krb5kdc[25075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837459, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM
WARNING: no policy specified for oryx/127.18.134.3@KRBTEST.COM; defaulting to no policy
I20260619 02:50:59.498772 25311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:59.480324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:48451 (local address 127.18.134.62:33075)
0619 02:50:59.480469 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:59.480474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:59.481738 (+ 1264us) server_negotiation.cc:407] Connection header received
0619 02:50:59.484151 (+ 2413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:59.484155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:59.484242 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:59.484324 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:59.486164 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:59.487003 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:59.488828 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:59.489392 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:59.489522 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:59.493001 (+ 3479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:59.493006 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:59.493010 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:59.493041 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:59.495452 (+ 2411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:59.496319 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:59.496323 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:59.496325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:59.496461 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:59.496965 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:59.496970 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:59.496973 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:59.497322 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:59.497437 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:59.498298 (+ 861us) server_negotiation.cc:299] Negotiation successful
0619 02:50:59.498472 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 02:50:59.499913 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:59.480743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.18.134.2:48451)
0619 02:50:59.481569 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:59.481618 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:59.483875 (+ 2257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:59.484569 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:59.484580 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:59.485209 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:59.485930 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:59.485953 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:59.487186 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:59.487191 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:59.488669 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:59.488679 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:59.489617 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:59.489624 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:59.489874 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:59.490514 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:59.490535 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:59.492798 (+ 2263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:59.495586 (+ 2788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:59.495595 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:59.495613 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:59.496175 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:59.496575 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:59.496580 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:59.496584 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:59.496819 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:59.497450 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:59.497458 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:59.497900 (+ 442us) client_negotiation.cc:769] Sending connection context
0619 02:50:59.498229 (+ 329us) client_negotiation.cc:240] Negotiation successful
0619 02:50:59.498786 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":332,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.134.3@KRBTEST.COM" created.
I20260619 02:50:59.501430 25443 heartbeater.cc:347] Connected to a master server at 127.18.134.62:33075
I20260619 02:50:59.501798 25443 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:59.502657 25443 heartbeater.cc:511] Master 127.18.134.62:33075 requested a full tablet report, sending...
I20260619 02:50:59.504163 25120 ts_manager.cc:194] Registered new tserver with Master: f46cfab95e6e44c0be837c69674063c9 (127.18.134.2:45195)
I20260619 02:50:59.505151 25120 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.134.2@KRBTEST.COM'} at 127.18.134.2:48451
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.134.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.3.keytab.
Entry for principal oryx/127.18.134.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.3.keytab.
I20260619 02:50:59.523506 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:0
--local_ip_for_outbound_sockets=127.18.134.3
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:33075
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.3.keytab
--principal=oryx/127.18.134.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:35501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:50:59.834863 25448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:50:59.835157 25448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:50:59.835249 25448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:50:59.842656 25448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:50:59.842780 25448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:50:59.842902 25448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:50:59.852821 25448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:35501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.3.keytab
--principal=oryx/127.18.134.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:33075
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:50:59.854055 25448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:50:59.855336 25448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:50:59.867853 25453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:59.867916 25454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:50:59.867885 25456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:50:59.869002 25448 server_base.cc:1034] running on GCE node
I20260619 02:50:59.869642 25448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:50:59.870600 25448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:50:59.871842 25448 hybrid_clock.cc:648] HybridClock initialized: now 1781837459871786 us; error 56 us; skew 500 ppm
I20260619 02:50:59.872150 25448 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:50:59 dist-test-slave-9mr0 krb5kdc[25075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837459, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:50:59.875479 25448 init.cc:377] Logged in from keytab as oryx/127.18.134.3@KRBTEST.COM (short username oryx)
I20260619 02:50:59.876940 25448 webserver.cc:460] Webserver started at http://127.18.134.3:36349/ using document root <none> and password file <none>
I20260619 02:50:59.877606 25448 fs_manager.cc:359] Metadata directory not provided
I20260619 02:50:59.877719 25448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:50:59.878000 25448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:50:59.880908 25448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/data/instance:
uuid: "7c852f00331741bf97c3d710d873c775"
format_stamp: "Formatted at 2026-06-19 02:50:59 on dist-test-slave-9mr0"
server_key: "c7b20881c45f2973ea33ca58b71ff7f8"
server_key_iv: "5223e859469b4f6305982b1ed4fd6fd3"
server_key_version: "encryptionkey@0"
I20260619 02:50:59.881654 25448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance:
uuid: "7c852f00331741bf97c3d710d873c775"
format_stamp: "Formatted at 2026-06-19 02:50:59 on dist-test-slave-9mr0"
server_key: "c7b20881c45f2973ea33ca58b71ff7f8"
server_key_iv: "5223e859469b4f6305982b1ed4fd6fd3"
server_key_version: "encryptionkey@0"
I20260619 02:50:59.886699 25448 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 02:50:59.890307 25463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:59.891830 25448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260619 02:50:59.891997 25448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "7c852f00331741bf97c3d710d873c775"
format_stamp: "Formatted at 2026-06-19 02:50:59 on dist-test-slave-9mr0"
server_key: "c7b20881c45f2973ea33ca58b71ff7f8"
server_key_iv: "5223e859469b4f6305982b1ed4fd6fd3"
server_key_version: "encryptionkey@0"
I20260619 02:50:59.892179 25448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:50:59.917690 25448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:50:59.922202 25448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:50:59.922462 25448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:50:59.923442 25448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:50:59.924723 25448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:50:59.924791 25448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:59.924898 25448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:50:59.924944 25448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:50:59.952798 25448 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:34939
I20260619 02:50:59.952877 25575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:34939 every 8 connection(s)
I20260619 02:50:59.954034 25448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:50:59.961360 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 25448
I20260619 02:50:59.961537 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance
I20260619 02:50:59.962076 18968 external_mini_cluster.cc:1428] Setting key ed9822abee750359c019e0729d35ddd2
Jun 19 02:50:59 dist-test-slave-9mr0 krb5kdc[25075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837459, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM
I20260619 02:50:59.973372 25311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:59.956547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:34451 (local address 127.18.134.62:33075)
0619 02:50:59.956752 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 02:50:59.956759 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:50:59.957853 (+ 1094us) server_negotiation.cc:407] Connection header received
0619 02:50:59.959646 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:50:59.959651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:50:59.959719 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:50:59.959806 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:50:59.961231 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:59.962313 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:59.964097 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:50:59.964670 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:50:59.964788 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:59.968106 (+ 3318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:50:59.968111 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:50:59.968114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:50:59.968145 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:50:59.970491 (+ 2346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:59.971219 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:59.971224 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:59.971228 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:59.971379 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:50:59.971865 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:50:59.971869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:50:59.971872 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:50:59.972159 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:50:59.972284 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 02:50:59.972974 (+ 690us) server_negotiation.cc:299] Negotiation successful
0619 02:50:59.973129 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260619 02:50:59.974352 25578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:50:59.957008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.18.134.3:34451)
0619 02:50:59.957686 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0619 02:50:59.957731 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 02:50:59.959377 (+ 1646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:50:59.960003 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:50:59.960011 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:50:59.960507 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:50:59.961071 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:59.961087 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:59.962454 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:59.962458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:59.963979 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:50:59.963990 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:50:59.964866 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:50:59.964871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:50:59.965128 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:50:59.965726 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:50:59.965750 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:50:59.967889 (+ 2139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:50:59.970621 (+ 2732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:59.970662 (+ 41us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:59.970680 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:59.971094 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:59.971513 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:50:59.971517 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:50:59.971519 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:50:59.971720 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:50:59.972305 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:50:59.972325 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:50:59.972636 (+ 311us) client_negotiation.cc:769] Sending connection context
0619 02:50:59.972865 (+ 229us) client_negotiation.cc:240] Negotiation successful
0619 02:50:59.973246 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":261,"threads_started":1}
I20260619 02:50:59.975754 25576 heartbeater.cc:347] Connected to a master server at 127.18.134.62:33075
I20260619 02:50:59.976042 25576 heartbeater.cc:464] Registering TS with master...
I20260619 02:50:59.976804 25576 heartbeater.cc:511] Master 127.18.134.62:33075 requested a full tablet report, sending...
I20260619 02:50:59.978152 25120 ts_manager.cc:194] Registered new tserver with Master: 7c852f00331741bf97c3d710d873c775 (127.18.134.3:34939)
I20260619 02:50:59.979022 25120 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.134.3@KRBTEST.COM'} at 127.18.134.3:34451
I20260619 02:50:59.989331 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.134.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.134.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.134.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.61.keytab.
Entry for principal oryx/127.18.134.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.61.keytab.
I20260619 02:51:00.032380 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.61:35581
--webserver_interface=127.18.134.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.61.keytab
--principal=oryx/127.18.134.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:35501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:33075,127.18.134.61:35581
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:51:00.033663 25309 heartbeater.cc:503] Master 127.18.134.62:33075 was elected leader, sending a full tablet report...
W20260619 02:51:00.343675 25581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:00.343962 25581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:00.344085 25581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:00.351290 25581 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:51:00.351414 25581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:00.351490 25581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:00.351564 25581 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:51:00.351624 25581 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:51:00.361354 25581 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:35501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:33075,127.18.134.61:35581
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.61.keytab
--principal=oryx/127.18.134.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.61:35581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.134.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
W20260619 02:51:00.361716 25581 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.18.134.62:33075,127.18.134.61:35581'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260619 02:51:00.362716 25581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:00.363988 25581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:00.375164 25587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:00.375165 25589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:00.375186 25586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:00.375901 25581 server_base.cc:1034] running on GCE node
I20260619 02:51:00.376667 25581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:00.378073 25581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:00.379276 25581 hybrid_clock.cc:648] HybridClock initialized: now 1781837460379211 us; error 42 us; skew 500 ppm
I20260619 02:51:00.379643 25581 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:51:00 dist-test-slave-9mr0 krb5kdc[25075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837460, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:51:00.382670 25581 init.cc:377] Logged in from keytab as oryx/127.18.134.61@KRBTEST.COM (short username oryx)
I20260619 02:51:00.384173 25581 webserver.cc:460] Webserver started at http://127.18.134.61:35645/ using document root <none> and password file <none>
I20260619 02:51:00.384847 25581 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:00.384930 25581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:00.385207 25581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:51:00.388197 25581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-1/data/instance:
uuid: "9a94a362219f40ec98932b3e1f58a9cd"
format_stamp: "Formatted at 2026-06-19 02:51:00 on dist-test-slave-9mr0"
server_key: "f59e0f37ec54571a9b96bea3e3484131"
server_key_iv: "12f76fad806641cb8b0b07564bea8835"
server_key_version: "encryptionkey@0"
I20260619 02:51:00.388913 25581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-1/wal/instance:
uuid: "9a94a362219f40ec98932b3e1f58a9cd"
format_stamp: "Formatted at 2026-06-19 02:51:00 on dist-test-slave-9mr0"
server_key: "f59e0f37ec54571a9b96bea3e3484131"
server_key_iv: "12f76fad806641cb8b0b07564bea8835"
server_key_version: "encryptionkey@0"
I20260619 02:51:00.394090 25581 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260619 02:51:00.397522 25596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:00.398996 25581 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260619 02:51:00.399142 25581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-1/wal
uuid: "9a94a362219f40ec98932b3e1f58a9cd"
format_stamp: "Formatted at 2026-06-19 02:51:00 on dist-test-slave-9mr0"
server_key: "f59e0f37ec54571a9b96bea3e3484131"
server_key_iv: "12f76fad806641cb8b0b07564bea8835"
server_key_version: "encryptionkey@0"
I20260619 02:51:00.399345 25581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:00.416424 25581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:00.420704 25581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:00.420948 25581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:00.438303 25581 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.61:35581
I20260619 02:51:00.438336 25647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.61:35581 every 8 connection(s)
I20260619 02:51:00.439630 25581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-1/data/info.pb
I20260619 02:51:00.441166 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 25581
I20260619 02:51:00.444175 25648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:00.452998 25648 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:51:00 dist-test-slave-9mr0 krb5kdc[25075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.134.61@KRBTEST.COM
Jun 19 02:51:00 dist-test-slave-9mr0 krb5kdc[25075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837460, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.61@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM
I20260619 02:51:00.465967 25651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:00.443837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50466 (local address 127.18.134.61:35581)
0619 02:51:00.444599 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:00.444614 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:00.444678 (+ 64us) server_negotiation.cc:407] Connection header received
0619 02:51:00.447087 (+ 2409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:00.447116 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:00.447649 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:00.448135 (+ 486us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:00.449756 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:00.451739 (+ 1983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:00.453556 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:00.454513 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:00.454972 (+ 459us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:00.459027 (+ 4055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:00.459033 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:00.459042 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:00.459075 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:00.462054 (+ 2979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:00.462740 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:00.462749 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:00.462758 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:00.462925 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:00.463383 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:00.463387 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:00.463389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:00.463983 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:00.464278 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:00.464619 (+ 341us) server_negotiation.cc:299] Negotiation successful
0619 02:51:00.464988 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":322,"threads_started":1}
I20260619 02:51:00.470016 25311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:00.455960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37732 (local address 127.18.134.62:33075)
0619 02:51:00.456131 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:00.456137 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:00.456818 (+ 681us) server_negotiation.cc:407] Connection header received
0619 02:51:00.457104 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:00.457110 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:00.457189 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:00.457287 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:00.458235 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:00.458984 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:00.460527 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:00.461031 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:00.461172 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:00.464636 (+ 3464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:00.464643 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:00.464648 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:00.464686 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:00.467516 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:00.468357 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:00.468361 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:00.468363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:00.468490 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:00.468857 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:00.468862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:00.468865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:00.469183 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:00.469287 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:00.469637 (+ 350us) server_negotiation.cc:299] Negotiation successful
0619 02:51:00.469784 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 02:51:00.470082 25652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:00.455997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.0.0.1:37732)
0619 02:51:00.456693 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:00.456738 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:00.456930 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:00.457352 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:00.457357 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:00.457884 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:00.458108 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:00.458115 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:00.459116 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:00.459120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:00.460385 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:00.460397 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:00.461247 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:00.461253 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:00.461407 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:00.462070 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:00.462088 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:00.464443 (+ 2355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:00.467641 (+ 3198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:00.467650 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:00.467656 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:00.468238 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:00.468591 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:00.468596 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:00.468599 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:00.468741 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:00.469283 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:00.469290 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:00.469424 (+ 134us) client_negotiation.cc:769] Sending connection context
0619 02:51:00.469565 (+ 141us) client_negotiation.cc:240] Negotiation successful
0619 02:51:00.469778 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":338,"threads_started":1}
I20260619 02:51:00.471530 25648 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:51:00 dist-test-slave-9mr0 krb5kdc[25075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837460, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.61@KRBTEST.COM for oryx/127.18.134.61@KRBTEST.COM
I20260619 02:51:00.482972 25652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:00.472154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:35581 (local address 127.0.0.1:50472)
0619 02:51:00.472311 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:00.472325 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:00.472411 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:00.472661 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:00.472665 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:00.473000 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:00.473127 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:00.473135 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:00.474044 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:00.474047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:00.474917 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:00.474927 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:00.475545 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:00.475550 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:00.475695 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:00.476209 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:00.476224 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:00.478236 (+ 2012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:00.480902 (+ 2666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:00.480906 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:00.480909 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:00.481256 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:00.481592 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:00.481596 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:00.481598 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:00.481726 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:00.482248 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:00.482252 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:00.482374 (+ 122us) client_negotiation.cc:769] Sending connection context
0619 02:51:00.482482 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 02:51:00.482750 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 02:51:00.483248 25651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:00.472324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50472 (local address 127.18.134.61:35581)
0619 02:51:00.472457 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:00.472462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:00.472481 (+ 19us) server_negotiation.cc:407] Connection header received
0619 02:51:00.472517 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:00.472521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:00.472567 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:00.472637 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:00.473242 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:00.473924 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:00.475030 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:00.475424 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:00.475521 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:00.478395 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:00.478401 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:00.478404 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:00.478430 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:00.480772 (+ 2342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:00.481347 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:00.481351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:00.481353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:00.481472 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:00.481838 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:00.481842 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:00.481844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:00.482119 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:00.482230 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:00.482606 (+ 376us) server_negotiation.cc:299] Negotiation successful
0619 02:51:00.482904 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":41}
I20260619 02:51:00.487534 25648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd: Bootstrap starting.
I20260619 02:51:00.490756 25648 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:00.491748 25648 log.cc:824] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd: Log is configured to *not* fsync() on all Append() calls
I20260619 02:51:00.494424 25648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd: No bootstrap required, opened a new log
I20260619 02:51:00.501279 25648 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } }
I20260619 02:51:00.501600 25648 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:00.501693 25648 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a94a362219f40ec98932b3e1f58a9cd, State: Initialized, Role: FOLLOWER
I20260619 02:51:00.502249 25648 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } }
I20260619 02:51:00.503063 25654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } } }
I20260619 02:51:00.503407 25654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:51:00.504114 25648 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:51:00.507881 25443 heartbeater.cc:503] Master 127.18.134.62:33075 was elected leader, sending a full tablet report...
W20260619 02:51:00.512355 25665 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:51:00.512490 25665 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:51:00.514302 25581 master_runner.cc:428] Detected that this master 9a94a362219f40ec98932b3e1f58a9cd is joining an existing cluster
I20260619 02:51:00.514392 25581 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.134.61:35581
I20260619 02:51:00.533849 25651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:00.518151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50480 (local address 127.18.134.61:35581)
0619 02:51:00.518329 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:00.518335 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:00.518927 (+ 592us) server_negotiation.cc:407] Connection header received
0619 02:51:00.519129 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:00.519135 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:00.519213 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:00.519325 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:00.521018 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:00.522063 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:00.523996 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:00.524597 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:00.524703 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:00.527292 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:00.527298 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:00.527303 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:00.527334 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:00.531023 (+ 3689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:00.531763 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:00.531768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:00.531771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:00.531915 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:00.532316 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:00.532321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:00.532324 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:00.532658 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:00.532807 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:00.533262 (+ 455us) server_negotiation.cc:299] Negotiation successful
0619 02:51:00.533460 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":58}
I20260619 02:51:00.534772 25311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:00.517801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37744 (local address 127.18.134.62:33075)
0619 02:51:00.517957 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:00.517961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:00.519591 (+ 1630us) server_negotiation.cc:407] Connection header received
0619 02:51:00.519713 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:00.519718 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:00.519801 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:00.519892 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:00.521172 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:00.522244 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:00.525028 (+ 2784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:00.525633 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:00.525727 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:00.528149 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:00.528155 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:00.528161 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:00.528217 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:00.531353 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:00.532629 (+ 1276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:00.532634 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:00.532637 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:00.532789 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:00.533180 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:00.533185 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:00.533188 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:00.533511 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:00.533636 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:00.534174 (+ 538us) server_negotiation.cc:299] Negotiation successful
0619 02:51:00.534406 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 02:51:00.535104 25671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:00.517977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:35581 (local address 127.0.0.1:50480)
0619 02:51:00.518806 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:00.518826 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:00.518981 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:00.519765 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:00.519769 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:00.520433 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:00.520850 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:00.520862 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:00.522243 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:00.522248 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:00.523849 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:00.523863 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:00.524770 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:00.524776 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:00.524950 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:00.525591 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:00.525613 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:00.527115 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:00.531154 (+ 4039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:00.531159 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:00.531163 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:00.531636 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:00.532027 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:00.532031 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:00.532038 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:00.532216 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:00.532868 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:00.532872 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:00.533032 (+ 160us) client_negotiation.cc:769] Sending connection context
0619 02:51:00.534216 (+ 1184us) client_negotiation.cc:240] Negotiation successful
0619 02:51:00.534479 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"mutex_wait_us":5,"thread_start_us":486,"threads_started":1}
I20260619 02:51:00.535773 25672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:00.517984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.0.0.1:37744)
0619 02:51:00.519495 (+ 1511us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:00.519512 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:00.519616 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:00.520049 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:00.520053 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:00.520662 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:00.521029 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:00.521038 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:00.522427 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:00.522432 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:00.524879 (+ 2447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:00.524892 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:00.526043 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:00.526051 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:00.526252 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:00.526845 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:00.526870 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:00.527953 (+ 1083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:00.532057 (+ 4104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:00.532060 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:00.532063 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:00.532505 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:00.532909 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:00.532913 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:00.532916 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:00.533062 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:00.533706 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:00.533710 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:00.533895 (+ 185us) client_negotiation.cc:769] Sending connection context
0619 02:51:00.535053 (+ 1158us) client_negotiation.cc:240] Negotiation successful
0619 02:51:00.535379 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1400,"mutex_wait_us":85,"thread_start_us":1158,"threads_started":1}
WARNING: no policy specified for oryx/127.18.134.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.134.60@KRBTEST.COM" created.
Jun 19 02:51:00 dist-test-slave-9mr0 krb5kdc[25075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837458, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.62@KRBTEST.COM for oryx/127.18.134.61@KRBTEST.COM
I20260619 02:51:00.557535 25651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:00.541267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50484 (local address 127.18.134.61:35581)
0619 02:51:00.541414 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:00.541420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:00.543284 (+ 1864us) server_negotiation.cc:407] Connection header received
0619 02:51:00.543604 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:00.543608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:00.543702 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:00.543780 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:00.544780 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:00.545556 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:00.546908 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:00.547349 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:00.547473 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:00.551429 (+ 3956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:00.551435 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:00.551440 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:00.551478 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:00.554907 (+ 3429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:00.555589 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:00.555593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:00.555595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:00.555731 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:00.556076 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:00.556080 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:00.556082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:00.556386 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:00.556517 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:00.556980 (+ 463us) server_negotiation.cc:299] Negotiation successful
0619 02:51:00.557179 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 02:51:00.557585 25674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:00.541249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:35581 (local address 127.0.0.1:50484)
0619 02:51:00.543157 (+ 1908us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:00.543203 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:00.543402 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:00.543841 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:00.543849 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:00.544448 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:00.544633 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:00.544641 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:00.545671 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:00.545677 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:00.546787 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:00.546798 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:00.547511 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:00.547517 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:00.547692 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:00.548361 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:00.548384 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:00.551241 (+ 2857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:00.555037 (+ 3796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:00.555049 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:00.555057 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:00.555464 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:00.555834 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:00.555839 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:00.555844 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:00.555972 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:00.556507 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:00.556515 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:00.556697 (+ 182us) client_negotiation.cc:769] Sending connection context
0619 02:51:00.556859 (+ 162us) client_negotiation.cc:240] Negotiation successful
0619 02:51:00.557179 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1703,"thread_start_us":1558,"threads_started":1}
I20260619 02:51:00.559799 25119 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.134.61:35581: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd"
member_type: NON_VOTER
last_known_addr {
host: "127.18.134.61"
port: 35581
}
attrs {
promote: true
}
}
dest_uuid: "0094edcfb4284971a6d45b1ec4e48a30"
cas_config_opid_index: -1
I20260619 02:51:00.560715 25119 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: NON_VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: true } }
I20260619 02:51:00.570434 25677 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 9a94a362219f40ec98932b3e1f58a9cd (127.18.134.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: NON_VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: true } } }
I20260619 02:51:00.571161 25623 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:00.575201 25677 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.134.61:35581
I20260619 02:51:00.575666 25678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0094edcfb4284971a6d45b1ec4e48a30" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: NON_VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: true } } }
I20260619 02:51:00.576016 25581 master.cc:537] Master@127.18.134.61:35581 shutting down...
I20260619 02:51:00.576071 25678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [sys.catalog]: This master's current role is: LEADER
I20260619 02:51:00.577147 25623 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [term 1 FOLLOWER]: Refusing update from remote peer 0094edcfb4284971a6d45b1ec4e48a30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260619 02:51:00.579280 25677 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: NON_VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.134.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.60.keytab.
Entry for principal oryx/127.18.134.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.60.keytab.
I20260619 02:51:00.582511 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.60:44329
--webserver_interface=127.18.134.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.60.keytab
--principal=oryx/127.18.134.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:35501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:33075,127.18.134.61:35581,127.18.134.60:44329
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:51:00.583320 25106 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260619 02:51:00.588883 25581 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:51:00.589319 25581 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:51:00.589493 25581 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd: stopping tablet replica
W20260619 02:51:00.591867 25106 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 -> Peer 9a94a362219f40ec98932b3e1f58a9cd (127.18.134.61:35581): Couldn't send request to peer 9a94a362219f40ec98932b3e1f58a9cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 02:51:00.605813 25581 master.cc:552] Master@127.18.134.61:35581 shutdown complete.
I20260619 02:51:00.605904 25581 master_runner.cc:305] Clearing existing system tablet
I20260619 02:51:00.606330 25581 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:00.606482 25581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:00.611274 25581 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260619 02:51:00.612893 25683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:00.613265 25581 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:00.613692 25581 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 02:51:00.613778 25581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-1/wal
uuid: "9a94a362219f40ec98932b3e1f58a9cd"
format_stamp: "Formatted at 2026-06-19 02:51:00 on dist-test-slave-9mr0"
server_key: "f59e0f37ec54571a9b96bea3e3484131"
server_key_iv: "12f76fad806641cb8b0b07564bea8835"
server_key_version: "encryptionkey@0"
I20260619 02:51:00.618570 25581 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:51:00.625015 25581 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 02:51:00.625193 25581 log.cc:1196] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260619 02:51:00.625610 25581 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd: Deleting consensus metadata
I20260619 02:51:00.625864 25581 master_runner.cc:315] Copying system tablet from 127.18.134.62:33075
I20260619 02:51:00.629825 25581 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd: tablet copy: Beginning tablet copy session from remote peer at address 127.18.134.62:33075
I20260619 02:51:00.645049 25311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:00.630603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37760 (local address 127.18.134.62:33075)
0619 02:51:00.630831 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:00.630837 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:00.631327 (+ 490us) server_negotiation.cc:407] Connection header received
0619 02:51:00.631517 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:00.631523 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:00.631617 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:00.631724 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:00.632724 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:00.633825 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:00.635601 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:00.636256 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:00.636382 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:00.638598 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:00.638605 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:00.638610 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:00.638678 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:00.642026 (+ 3348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:00.642969 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:00.642975 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:00.642979 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:00.643121 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:00.643535 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:00.643540 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:00.643543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:00.643907 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:00.644052 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:00.644477 (+ 425us) server_negotiation.cc:299] Negotiation successful
0619 02:51:00.644712 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260619 02:51:00.645110 25689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:00.630544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.0.0.1:37760)
0619 02:51:00.631203 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:00.631223 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:00.631372 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:00.631778 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:00.631783 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:00.632294 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:00.632562 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:00.632574 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:00.633972 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:00.633977 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:00.635452 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:00.635468 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:00.636414 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:00.636419 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:00.636608 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:00.637314 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:00.637339 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:00.638378 (+ 1039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:00.642177 (+ 3799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:00.642183 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:00.642187 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:00.642850 (+ 663us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:00.643240 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:00.643245 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:00.643249 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:00.643424 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:00.644032 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:00.644037 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:00.644208 (+ 171us) client_negotiation.cc:769] Sending connection context
0619 02:51:00.644357 (+ 149us) client_negotiation.cc:240] Negotiation successful
0619 02:51:00.644679 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":390,"threads_started":1}
I20260619 02:51:00.646171 25140 tablet_copy_service.cc:140] P 0094edcfb4284971a6d45b1ec4e48a30: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9a94a362219f40ec98932b3e1f58a9cd ({username='oryx', principal='oryx/127.18.134.61@KRBTEST.COM'} at 127.0.0.1:37760)
I20260619 02:51:00.646476 25140 tablet_copy_service.cc:161] P 0094edcfb4284971a6d45b1ec4e48a30: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9a94a362219f40ec98932b3e1f58a9cd at {username='oryx', principal='oryx/127.18.134.61@KRBTEST.COM'} at 127.0.0.1:37760: session id = 9a94a362219f40ec98932b3e1f58a9cd-00000000000000000000000000000000
I20260619 02:51:00.650913 25140 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 02:51:00.653230 25581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 02:51:00.659870 25581 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd: tablet copy: Starting download of 0 data blocks...
I20260619 02:51:00.660184 25581 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd: tablet copy: Starting download of 1 WAL segments...
I20260619 02:51:00.662972 25581 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 02:51:00.667316 25140 tablet_copy_service.cc:342] P 0094edcfb4284971a6d45b1ec4e48a30: Request end of tablet copy session 9a94a362219f40ec98932b3e1f58a9cd-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.134.61@KRBTEST.COM'} at 127.0.0.1:37760
I20260619 02:51:00.667567 25140 tablet_copy_service.cc:433] P 0094edcfb4284971a6d45b1ec4e48a30: ending tablet copy session 9a94a362219f40ec98932b3e1f58a9cd-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9a94a362219f40ec98932b3e1f58a9cd
I20260619 02:51:00.679541 25581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:00.682169 25690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:00.683940 25691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:00.694929 25581 server_base.cc:1034] running on GCE node
W20260619 02:51:00.696137 25693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:00.696597 25581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:00.697309 25581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:00.698581 25581 hybrid_clock.cc:648] HybridClock initialized: now 1781837460698520 us; error 36 us; skew 500 ppm
I20260619 02:51:00.698886 25581 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:51:00 dist-test-slave-9mr0 krb5kdc[25075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837460, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:51:00.701041 25581 init.cc:377] Logged in from keytab as oryx/127.18.134.61@KRBTEST.COM (short username oryx)
I20260619 02:51:00.702346 25581 webserver.cc:460] Webserver started at http://127.18.134.61:43437/ using document root <none> and password file <none>
I20260619 02:51:00.702741 25581 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:00.702827 25581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:00.705870 25581 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260619 02:51:00.707851 25700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:00.708268 25581 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:51:00.708408 25581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-1/wal
uuid: "9a94a362219f40ec98932b3e1f58a9cd"
format_stamp: "Formatted at 2026-06-19 02:51:00 on dist-test-slave-9mr0"
server_key: "f59e0f37ec54571a9b96bea3e3484131"
server_key_iv: "12f76fad806641cb8b0b07564bea8835"
server_key_version: "encryptionkey@0"
I20260619 02:51:00.708568 25581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:00.739766 25581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:00.743606 25581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:00.764600 25581 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.61:35581
I20260619 02:51:00.764647 25751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.61:35581 every 8 connection(s)
I20260619 02:51:00.765574 25581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-1/data/info.pb
I20260619 02:51:00.770792 25752 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:51:00.774511 25752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd: Bootstrap starting.
I20260619 02:51:00.805933 25752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:51:00.806461 25752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd: Bootstrap complete.
I20260619 02:51:00.807533 25752 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: NON_VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: true } }
I20260619 02:51:00.807832 25752 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9a94a362219f40ec98932b3e1f58a9cd, State: Initialized, Role: LEARNER
I20260619 02:51:00.808120 25752 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: NON_VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: true } }
I20260619 02:51:00.808984 25755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: NON_VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: true } } }
I20260619 02:51:00.809335 25752 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:51:00.809377 25755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [sys.catalog]: This master's current role is: LEARNER
I20260619 02:51:00.821101 25766 catalog_manager.cc:1259] Loaded cluster ID: 9cbe8949e88b4785a8e697bf8dceef26
I20260619 02:51:00.821228 25766 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd: loading cluster ID for follower catalog manager: success
I20260619 02:51:00.824854 25766 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd: acquiring CA information for follower catalog manager: success
I20260619 02:51:00.827280 25766 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 02:51:00.949295 25681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:00.949621 25681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:00.949739 25681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:00.957170 25681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:51:00.957291 25681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:00.957362 25681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:00.957440 25681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:51:00.957499 25681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:51:00.968047 25681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:35501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:33075,127.18.134.61:35581,127.18.134.60:44329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.60.keytab
--principal=oryx/127.18.134.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.60:44329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.134.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:00.969390 25681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:00.970726 25681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:51:00.981391 25576 heartbeater.cc:503] Master 127.18.134.62:33075 was elected leader, sending a full tablet report...
W20260619 02:51:00.982161 25772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:00.982141 25774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:00.982367 25771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:00.983712 25681 server_base.cc:1034] running on GCE node
I20260619 02:51:00.984601 25681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:00.986096 25681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:00.987311 25681 hybrid_clock.cc:648] HybridClock initialized: now 1781837460987250 us; error 51 us; skew 500 ppm
I20260619 02:51:00.987725 25681 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:51:00 dist-test-slave-9mr0 krb5kdc[25075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837460, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:51:00.991062 25681 init.cc:377] Logged in from keytab as oryx/127.18.134.60@KRBTEST.COM (short username oryx)
I20260619 02:51:00.992877 25681 webserver.cc:460] Webserver started at http://127.18.134.60:33433/ using document root <none> and password file <none>
I20260619 02:51:00.993572 25681 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:00.993676 25681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:00.993974 25681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:51:00.997000 25681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-2/data/instance:
uuid: "59aa8b8f204d41a9b88df69b6eca76d3"
format_stamp: "Formatted at 2026-06-19 02:51:00 on dist-test-slave-9mr0"
server_key: "81d3e99cf8eb65b27d228c4d5f9b2681"
server_key_iv: "a7dd98181e8dc2886fa6cd1cc48f8d27"
server_key_version: "encryptionkey@0"
I20260619 02:51:00.997776 25681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-2/wal/instance:
uuid: "59aa8b8f204d41a9b88df69b6eca76d3"
format_stamp: "Formatted at 2026-06-19 02:51:00 on dist-test-slave-9mr0"
server_key: "81d3e99cf8eb65b27d228c4d5f9b2681"
server_key_iv: "a7dd98181e8dc2886fa6cd1cc48f8d27"
server_key_version: "encryptionkey@0"
I20260619 02:51:01.003221 25681 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 02:51:01.007211 25781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:01.008644 25681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 02:51:01.008803 25681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-2/wal
uuid: "59aa8b8f204d41a9b88df69b6eca76d3"
format_stamp: "Formatted at 2026-06-19 02:51:00 on dist-test-slave-9mr0"
server_key: "81d3e99cf8eb65b27d228c4d5f9b2681"
server_key_iv: "a7dd98181e8dc2886fa6cd1cc48f8d27"
server_key_version: "encryptionkey@0"
I20260619 02:51:01.008982 25681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:01.019557 25681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:01.023209 25681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:01.023451 25681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:01.042474 25681 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.60:44329
I20260619 02:51:01.042528 25833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.60:44329 every 8 connection(s)
I20260619 02:51:01.044042 25681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-2/data/info.pb
I20260619 02:51:01.044106 25819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.037354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50486 (local address 127.18.134.61:35581)
0619 02:51:01.037802 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:01.037807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:01.037821 (+ 14us) server_negotiation.cc:407] Connection header received
0619 02:51:01.037873 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:01.037878 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:01.037971 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:01.038093 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:01.038600 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.039959 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.042198 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.043227 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.043340 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.043489 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:01.043779 (+ 290us) server_negotiation.cc:299] Negotiation successful
0619 02:51:01.043875 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":235,"threads_started":1}
I20260619 02:51:01.044138 25674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.036763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:35581 (local address 127.0.0.1:50486)
0619 02:51:01.036987 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:01.037005 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:01.037107 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:01.038189 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:01.038195 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:01.038231 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:01.038432 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:01.038445 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.040100 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.040107 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:01.042027 (+ 1920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:01.042043 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.043403 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.043409 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:01.043596 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.043661 (+ 65us) client_negotiation.cc:769] Sending connection context
0619 02:51:01.043760 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 02:51:01.043889 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260619 02:51:01.045074 25727 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260619 02:51:01.045591 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 25681
I20260619 02:51:01.045679 25755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0094edcfb4284971a6d45b1ec4e48a30. Latest consensus state: current_term: 1 leader_uuid: "0094edcfb4284971a6d45b1ec4e48a30" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: NON_VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: true } } }
I20260619 02:51:01.045946 25755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [sys.catalog]: This master's current role is: LEARNER
I20260619 02:51:01.048636 25834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:01.056613 25834 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:51:01 dist-test-slave-9mr0 krb5kdc[25075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.134.60@KRBTEST.COM
Jun 19 02:51:01 dist-test-slave-9mr0 krb5kdc[25075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837460, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.60@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM
I20260619 02:51:01.071522 25837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.048265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47578 (local address 127.18.134.60:44329)
0619 02:51:01.049010 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:01.049021 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:01.049059 (+ 38us) server_negotiation.cc:407] Connection header received
0619 02:51:01.050578 (+ 1519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:01.050599 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:01.051064 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:01.051417 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:01.053181 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.054574 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.056303 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.057316 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.057484 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.063445 (+ 5961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:01.063456 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:01.063471 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:01.063519 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:01.066834 (+ 3315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.067589 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.067598 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.067607 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.067817 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.068196 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.068199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.068201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.068705 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:01.068917 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:01.069434 (+ 517us) server_negotiation.cc:299] Negotiation successful
0619 02:51:01.069852 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":338,"threads_started":1}
I20260619 02:51:01.075505 25311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.059984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37762 (local address 127.18.134.62:33075)
0619 02:51:01.060235 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:01.060241 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:01.061409 (+ 1168us) server_negotiation.cc:407] Connection header received
0619 02:51:01.061808 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:01.061814 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:01.061894 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:01.061985 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:01.063569 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.064708 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.066158 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.066862 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.066976 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.070488 (+ 3512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:01.070493 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:01.070496 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:01.070528 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:01.073102 (+ 2574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.073863 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.073866 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.073869 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.073979 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.074329 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.074332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.074334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.074591 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:01.074740 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:01.075104 (+ 364us) server_negotiation.cc:299] Negotiation successful
0619 02:51:01.075280 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 02:51:01.075508 25838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.060040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.0.0.1:37762)
0619 02:51:01.061222 (+ 1182us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:01.061292 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:01.061588 (+ 296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:01.062075 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:01.062083 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:01.063034 (+ 951us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:01.063392 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:01.063406 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.064905 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.064911 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:01.066019 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:01.066031 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.067032 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.067039 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:01.067249 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.067848 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:01.067865 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:01.070297 (+ 2432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:01.073233 (+ 2936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:01.073241 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:01.073247 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:01.073752 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:01.074083 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:01.074087 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:01.074090 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:01.074229 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:01.074738 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:01.074743 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:01.074881 (+ 138us) client_negotiation.cc:769] Sending connection context
0619 02:51:01.075025 (+ 144us) client_negotiation.cc:240] Negotiation successful
0619 02:51:01.075234 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":811,"thread_start_us":570,"threads_started":1}
I20260619 02:51:01.076732 25834 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:51:01 dist-test-slave-9mr0 krb5kdc[25075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837460, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.60@KRBTEST.COM for oryx/127.18.134.61@KRBTEST.COM
I20260619 02:51:01.088470 25838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.077292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:35581 (local address 127.0.0.1:50502)
0619 02:51:01.077472 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:01.077487 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:01.077568 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:01.077897 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:01.077902 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:01.078190 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:01.078322 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:01.078331 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.079402 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.079406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:01.080316 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:01.080327 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.081046 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.081052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:01.081171 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.081637 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:01.081652 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:01.083516 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:01.086232 (+ 2716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:01.086236 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:01.086239 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:01.086588 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:01.087027 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:01.087030 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:01.087033 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:01.087164 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:01.087602 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:01.087605 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:01.087842 (+ 237us) client_negotiation.cc:769] Sending connection context
0619 02:51:01.087980 (+ 138us) client_negotiation.cc:240] Negotiation successful
0619 02:51:01.088229 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 02:51:01.088521 25819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.077478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50502 (local address 127.18.134.61:35581)
0619 02:51:01.077647 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:01.077653 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:01.077670 (+ 17us) server_negotiation.cc:407] Connection header received
0619 02:51:01.077718 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:01.077721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:01.077789 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:01.077867 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:01.078461 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.079256 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.080456 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.080894 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.081034 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.083696 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:01.083701 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:01.083705 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:01.083735 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:01.086131 (+ 2396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.086780 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.086784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.086786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.086929 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.087234 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.087237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.087239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.087480 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:01.087587 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:01.088085 (+ 498us) server_negotiation.cc:299] Negotiation successful
0619 02:51:01.088283 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 02:51:01.089471 25834 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44329 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:51:01 dist-test-slave-9mr0 krb5kdc[25075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837460, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.60@KRBTEST.COM for oryx/127.18.134.60@KRBTEST.COM
I20260619 02:51:01.100943 25838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.090052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44329 (local address 127.0.0.1:47584)
0619 02:51:01.090261 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:01.090278 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:01.090386 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:01.090656 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:01.090660 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:01.091027 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:01.091178 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:01.091189 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.092084 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.092089 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:01.092899 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:01.092908 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.093520 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.093525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:01.093677 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.094291 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:01.094306 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:01.096623 (+ 2317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:01.099079 (+ 2456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:01.099082 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:01.099085 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:01.099433 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:01.099712 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:01.099714 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:01.099717 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:01.099852 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:01.100260 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:01.100263 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:01.100384 (+ 121us) client_negotiation.cc:769] Sending connection context
0619 02:51:01.100481 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 02:51:01.100654 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260619 02:51:01.101016 25837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.090109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47584 (local address 127.18.134.60:44329)
0619 02:51:01.090231 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:01.090236 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:01.090341 (+ 105us) server_negotiation.cc:407] Connection header received
0619 02:51:01.090478 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:01.090483 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:01.090535 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:01.090606 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:01.091319 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.091975 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.093015 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.093383 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.093484 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.096799 (+ 3315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:01.096806 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:01.096811 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:01.096840 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:01.098967 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.099517 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.099521 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.099523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.099631 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.099932 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.099935 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.099937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.100175 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:01.100278 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:01.100550 (+ 272us) server_negotiation.cc:299] Negotiation successful
0619 02:51:01.100774 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":28}
I20260619 02:51:01.105244 25834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3: Bootstrap starting.
I20260619 02:51:01.108192 25834 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:01.109150 25834 log.cc:824] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3: Log is configured to *not* fsync() on all Append() calls
I20260619 02:51:01.111662 25834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3: No bootstrap required, opened a new log
I20260619 02:51:01.118875 25834 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44329 } }
I20260619 02:51:01.119176 25834 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:01.119290 25834 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59aa8b8f204d41a9b88df69b6eca76d3, State: Initialized, Role: FOLLOWER
I20260619 02:51:01.119827 25834 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44329 } }
I20260619 02:51:01.120692 25840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44329 } } }
I20260619 02:51:01.120997 25840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:51:01.121603 25834 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [sys.catalog]: configured and running, proceeding with master startup.
W20260619 02:51:01.131299 25851 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:51:01.131436 25851 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:51:01.135567 25681 master_runner.cc:428] Detected that this master 59aa8b8f204d41a9b88df69b6eca76d3 is joining an existing cluster
I20260619 02:51:01.135704 25681 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.134.60:44329
I20260619 02:51:01.156056 25311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.140013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37772 (local address 127.18.134.62:33075)
0619 02:51:01.140198 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:01.140205 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:01.140872 (+ 667us) server_negotiation.cc:407] Connection header received
0619 02:51:01.141137 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:01.141144 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:01.141223 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:01.141304 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:01.142851 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.143968 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.146410 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.147052 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.147147 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.149602 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:01.149608 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:01.149613 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:01.149650 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:01.152851 (+ 3201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.153681 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.153686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.153690 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.153849 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.154318 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.154323 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.154327 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.154687 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:01.154813 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:01.155497 (+ 684us) server_negotiation.cc:299] Negotiation successful
0619 02:51:01.155722 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 02:51:01.157357 25857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.140107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.0.0.1:37772)
0619 02:51:01.140763 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:01.140784 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:01.140979 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:01.141496 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:01.141501 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:01.142283 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:01.142674 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:01.142688 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.144135 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.144141 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:01.146243 (+ 2102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:01.146259 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.147416 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.147426 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:01.147604 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.148303 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:01.148325 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:01.149402 (+ 1077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:01.153028 (+ 3626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:01.153033 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:01.153038 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:01.153540 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:01.153999 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:01.154004 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:01.154008 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:01.154195 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:01.154904 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:01.154908 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:01.155125 (+ 217us) client_negotiation.cc:769] Sending connection context
0619 02:51:01.156609 (+ 1484us) client_negotiation.cc:240] Negotiation successful
0619 02:51:01.156908 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":355,"threads_started":1}
I20260619 02:51:01.172444 25859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.140722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:35581 (local address 127.0.0.1:50510)
0619 02:51:01.151077 (+ 10355us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:01.151098 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:01.151455 (+ 357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:01.152055 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:01.152060 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:01.153021 (+ 961us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:01.153408 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:01.153421 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.154965 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.154970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:01.156715 (+ 1745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:01.156730 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.159624 (+ 2894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.159637 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:01.161886 (+ 2249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.162707 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:01.162728 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:01.163792 (+ 1064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:01.169251 (+ 5459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:01.169257 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:01.169261 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:01.169826 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:01.170262 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:01.170267 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:01.170271 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:01.170466 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:01.171198 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:01.171203 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:01.171473 (+ 270us) client_negotiation.cc:769] Sending connection context
0619 02:51:01.171611 (+ 138us) client_negotiation.cc:240] Negotiation successful
0619 02:51:01.171948 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10190,"mutex_wait_us":29,"spinlock_wait_cycles":25472,"thread_start_us":19654,"threads_started":1}
I20260619 02:51:01.173205 25857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.140977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44329 (local address 127.0.0.1:47598)
0619 02:51:01.157793 (+ 16816us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:01.157810 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:01.157933 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:01.158331 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:01.158336 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:01.158812 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:01.160079 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:01.160091 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.161639 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.161644 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:01.163201 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:01.163214 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.164358 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.164363 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:01.164515 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.165044 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:01.165070 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:01.166131 (+ 1061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:01.170291 (+ 4160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:01.170295 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:01.170298 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:01.170811 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:01.171525 (+ 714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:01.171530 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:01.171533 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:01.171660 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:01.172289 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:01.172294 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:01.172446 (+ 152us) client_negotiation.cc:769] Sending connection context
0619 02:51:01.172606 (+ 160us) client_negotiation.cc:240] Negotiation successful
0619 02:51:01.172865 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16724,"mutex_wait_us":3,"thread_start_us":10259,"threads_started":1}
I20260619 02:51:01.173399 25819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.140143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50510 (local address 127.18.134.61:35581)
0619 02:51:01.140336 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:01.140342 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:01.151206 (+ 10864us) server_negotiation.cc:407] Connection header received
0619 02:51:01.151607 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:01.151614 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:01.151717 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:01.151790 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:01.153603 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.154754 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.156890 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.157585 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.157678 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.164015 (+ 6337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:01.164022 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:01.164026 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:01.164068 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:01.169100 (+ 5032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.169975 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.169981 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.169986 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.170135 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.170606 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.170611 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.170615 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.170994 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:01.171131 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:01.172868 (+ 1737us) server_negotiation.cc:299] Negotiation successful
0619 02:51:01.173095 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:51:01.175014 25837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.141427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47598 (local address 127.18.134.60:44329)
0619 02:51:01.141631 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:01.141636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:01.157867 (+ 16231us) server_negotiation.cc:407] Connection header received
0619 02:51:01.158037 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:01.158042 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:01.158100 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:01.158174 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:01.160255 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.161448 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.163356 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.164042 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.164130 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.166324 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:01.166330 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:01.166335 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:01.166367 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:01.170177 (+ 3810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.171247 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.171252 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.171255 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.171414 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.171757 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.171761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.171763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.172145 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:01.174485 (+ 2340us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:01.174602 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 02:51:01.174785 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Jun 19 02:51:01 dist-test-slave-9mr0 krb5kdc[25075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837458, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.62@KRBTEST.COM for oryx/127.18.134.60@KRBTEST.COM
I20260619 02:51:01.177791 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 25181
I20260619 02:51:01.183110 25674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.164146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44329 (local address 127.0.0.1:47606)
0619 02:51:01.164518 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:01.164536 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:01.164633 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:01.167846 (+ 3213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:01.167852 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:01.168274 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:01.168460 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:01.168473 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.170061 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.170067 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:01.171443 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:01.171459 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.172546 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.172553 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:01.172709 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.173574 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:01.173598 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:01.176796 (+ 3198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:01.180684 (+ 3888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:01.180690 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:01.180694 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:01.181199 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:01.181573 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:01.181578 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:01.181582 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:01.181702 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:01.182194 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:01.182199 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:01.182330 (+ 131us) client_negotiation.cc:769] Sending connection context
0619 02:51:01.182449 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 02:51:01.182749 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260}
I20260619 02:51:01.183159 25860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.164938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47606 (local address 127.18.134.60:44329)
0619 02:51:01.167440 (+ 2502us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:01.167449 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:01.167474 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:51:01.167568 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:01.167572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:01.167659 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:01.167775 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:01.168666 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.169909 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.171624 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.172365 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.176965 (+ 4600us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.177210 (+ 245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:01.177218 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:01.177223 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:01.177254 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:01.180543 (+ 3289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.181323 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.181328 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.181332 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.181460 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.181808 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.181813 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.181820 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.182075 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:01.182198 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:01.182556 (+ 358us) server_negotiation.cc:299] Negotiation successful
0619 02:51:01.182771 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":2352,"thread_start_us":2201,"threads_started":1}
I20260619 02:51:01.186235 25119 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.134.60:44329: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3"
member_type: NON_VOTER
last_known_addr {
host: "127.18.134.60"
port: 44329
}
attrs {
promote: true
}
}
dest_uuid: "0094edcfb4284971a6d45b1ec4e48a30"
cas_config_opid_index: 5
I20260619 02:51:01.187129 25119 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: NON_VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: true } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: NON_VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: true } }
I20260619 02:51:01.188947 25677 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 59aa8b8f204d41a9b88df69b6eca76d3 (127.18.134.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: NON_VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: true } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: NON_VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: true } } }
I20260619 02:51:01.189643 25808 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:01.190368 25727 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [term 1 LEARNER]: Refusing update from remote peer 0094edcfb4284971a6d45b1ec4e48a30: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
Jun 19 02:51:01 dist-test-slave-9mr0 krb5kdc[25075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.134.1@KRBTEST.COM
I20260619 02:51:01.193885 25808 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [term 1 FOLLOWER]: Refusing update from remote peer 0094edcfb4284971a6d45b1ec4e48a30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 02:51:01.194489 25677 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.134.60:44329
I20260619 02:51:01.195140 25681 master.cc:537] Master@127.18.134.60:44329 shutting down...
I20260619 02:51:01.198527 25864 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: NON_VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260619 02:51:01.206229 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.178816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49648 (local address 127.18.134.1:45679)
0619 02:51:01.179519 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:01.179528 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:01.179557 (+ 29us) server_negotiation.cc:407] Connection header received
0619 02:51:01.179620 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:01.179628 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:01.179912 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:01.180129 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:01.181391 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.182958 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.185458 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.186350 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.187207 (+ 857us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.192136 (+ 4929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:01.192151 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:01.192169 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:01.192222 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:01.200311 (+ 8089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.201262 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.201271 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.201275 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.201420 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.201883 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.201888 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.201892 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.202329 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:01.204843 (+ 2514us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:01.205053 (+ 210us) server_negotiation.cc:299] Negotiation successful
0619 02:51:01.205324 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":342,"threads_started":1}
I20260619 02:51:01.207876 25304 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:01.207835 25862 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: NON_VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:51:01.208415 25681 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:51:01.208873 25681 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:51:01.209141 25681 tablet_replica.cc:330] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3: stopping tablet replica
I20260619 02:51:01.214385 25864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0094edcfb4284971a6d45b1ec4e48a30" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: NON_VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: true } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: NON_VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: true } } }
I20260619 02:51:01.215207 25864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [sys.catalog]: This master's current role is: LEADER
I20260619 02:51:01.218248 25681 master.cc:552] Master@127.18.134.60:44329 shutdown complete.
I20260619 02:51:01.218374 25681 master_runner.cc:305] Clearing existing system tablet
I20260619 02:51:01.218850 25681 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:01.218989 25681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:01.223330 25681 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.002s
I20260619 02:51:01.223012 25727 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 59aa8b8f204d41a9b88df69b6eca76d3 (127.18.134.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: NON_VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: true } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: NON_VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: true } } }
I20260619 02:51:01.225821 25674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.225323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44329 (local address 127.0.0.1:47620)
0619 02:51:01.225501 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:01.225622 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260619 02:51:01.225978 25870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
W20260619 02:51:01.226166 25105 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 -> Peer 59aa8b8f204d41a9b88df69b6eca76d3 (127.18.134.60:44329): Couldn't send request to peer 59aa8b8f204d41a9b88df69b6eca76d3. Status: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44329: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 02:51:01.226588 25681 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:01.227022 25681 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260619 02:51:01.227182 25681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-2/wal
uuid: "59aa8b8f204d41a9b88df69b6eca76d3"
format_stamp: "Formatted at 2026-06-19 02:51:00 on dist-test-slave-9mr0"
server_key: "81d3e99cf8eb65b27d228c4d5f9b2681"
server_key_iv: "a7dd98181e8dc2886fa6cd1cc48f8d27"
server_key_version: "encryptionkey@0"
I20260619 02:51:01.228677 25755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0094edcfb4284971a6d45b1ec4e48a30" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: NON_VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: true } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: NON_VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: true } } }
I20260619 02:51:01.228968 25755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [sys.catalog]: This master's current role is: LEARNER
I20260619 02:51:01.230294 25863 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30: attempting to promote NON_VOTER 9a94a362219f40ec98932b3e1f58a9cd to VOTER
I20260619 02:51:01.231379 25863 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: NON_VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: true } }
I20260619 02:51:01.232313 25681 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:51:01.233872 25674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.233470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44329 (local address 127.0.0.1:47624)
0619 02:51:01.233605 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:01.233706 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 02:51:01.234058 25727 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [term 1 LEARNER]: Refusing update from remote peer 0094edcfb4284971a6d45b1ec4e48a30: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260619 02:51:01.234609 25862 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260619 02:51:01.234978 25674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.234510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44329 (local address 127.0.0.1:47632)
0619 02:51:01.234736 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:01.234818 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260619 02:51:01.235169 25105 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 -> Peer 59aa8b8f204d41a9b88df69b6eca76d3 (127.18.134.60:44329): Couldn't send request to peer 59aa8b8f204d41a9b88df69b6eca76d3. Status: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44329: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 02:51:01.237109 25864 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 9a94a362219f40ec98932b3e1f58a9cd (127.18.134.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: NON_VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: true } } }
I20260619 02:51:01.237808 25727 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 9a94a362219f40ec98932b3e1f58a9cd (127.18.134.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: NON_VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: true } } }
I20260619 02:51:01.241235 25677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0094edcfb4284971a6d45b1ec4e48a30" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: NON_VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: true } } }
I20260619 02:51:01.241201 25862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0094edcfb4284971a6d45b1ec4e48a30" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: NON_VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: true } } }
I20260619 02:51:01.241456 25862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [sys.catalog]: This master's current role is: LEADER
I20260619 02:51:01.241463 25677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [sys.catalog]: This master's current role is: LEADER
I20260619 02:51:01.243325 25755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0094edcfb4284971a6d45b1ec4e48a30" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: NON_VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: true } } }
I20260619 02:51:01.243551 25755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:51:01.245551 25681 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 02:51:01.245730 25681 log.cc:1196] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260619 02:51:01.246315 25681 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3: Deleting consensus metadata
I20260619 02:51:01.246548 25681 master_runner.cc:315] Copying system tablet from 127.18.134.62:33075
I20260619 02:51:01.252564 25681 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3: tablet copy: Beginning tablet copy session from remote peer at address 127.18.134.62:33075
I20260619 02:51:01.263952 25311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.253269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37784 (local address 127.18.134.62:33075)
0619 02:51:01.253505 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:01.253512 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:01.253916 (+ 404us) server_negotiation.cc:407] Connection header received
0619 02:51:01.254089 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:01.254095 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:01.254164 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:01.254242 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:01.255108 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.255839 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.256998 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.257402 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.257508 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.259156 (+ 1648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:01.259161 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:01.259166 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:01.259196 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:01.261553 (+ 2357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.262100 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.262106 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.262110 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.262229 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.262563 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.262568 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.262572 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.262873 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:01.262998 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:01.263404 (+ 406us) server_negotiation.cc:299] Negotiation successful
0619 02:51:01.263596 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260619 02:51:01.263981 25876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.253213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.0.0.1:37784)
0619 02:51:01.253815 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:01.253832 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:01.253965 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:01.254285 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:01.254294 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:01.254768 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:01.254972 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:01.254982 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.255968 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.255974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:01.256867 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:01.256879 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.257538 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.257544 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:01.257710 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.258244 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:01.258261 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:01.258990 (+ 729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:01.261670 (+ 2680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:01.261676 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:01.261680 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:01.261999 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:01.262327 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:01.262332 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:01.262336 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:01.262467 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:01.262979 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:01.262983 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:01.263145 (+ 162us) client_negotiation.cc:769] Sending connection context
0619 02:51:01.263276 (+ 131us) client_negotiation.cc:240] Negotiation successful
0619 02:51:01.263548 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":401,"threads_started":1}
I20260619 02:51:01.264935 25140 tablet_copy_service.cc:140] P 0094edcfb4284971a6d45b1ec4e48a30: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 59aa8b8f204d41a9b88df69b6eca76d3 ({username='oryx', principal='oryx/127.18.134.60@KRBTEST.COM'} at 127.0.0.1:37784)
I20260619 02:51:01.265100 25140 tablet_copy_service.cc:161] P 0094edcfb4284971a6d45b1ec4e48a30: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 59aa8b8f204d41a9b88df69b6eca76d3 at {username='oryx', principal='oryx/127.18.134.60@KRBTEST.COM'} at 127.0.0.1:37784: session id = 59aa8b8f204d41a9b88df69b6eca76d3-00000000000000000000000000000000
I20260619 02:51:01.267751 25140 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 02:51:01.269272 25681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 02:51:01.273672 25681 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3: tablet copy: Starting download of 0 data blocks...
I20260619 02:51:01.273931 25681 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3: tablet copy: Starting download of 1 WAL segments...
I20260619 02:51:01.275923 25681 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 02:51:01.278872 25140 tablet_copy_service.cc:342] P 0094edcfb4284971a6d45b1ec4e48a30: Request end of tablet copy session 59aa8b8f204d41a9b88df69b6eca76d3-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.134.60@KRBTEST.COM'} at 127.0.0.1:37784
I20260619 02:51:01.278983 25140 tablet_copy_service.cc:433] P 0094edcfb4284971a6d45b1ec4e48a30: ending tablet copy session 59aa8b8f204d41a9b88df69b6eca76d3-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 59aa8b8f204d41a9b88df69b6eca76d3
I20260619 02:51:01.287985 25681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:01.290388 25877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:01.296320 25878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:01.303584 25880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:01.304061 25681 server_base.cc:1034] running on GCE node
I20260619 02:51:01.304443 25681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:01.305073 25681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:01.306396 25681 hybrid_clock.cc:648] HybridClock initialized: now 1781837461306317 us; error 57 us; skew 500 ppm
I20260619 02:51:01.306677 25681 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:51:01 dist-test-slave-9mr0 krb5kdc[25075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837461, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:51:01.308370 25681 init.cc:377] Logged in from keytab as oryx/127.18.134.60@KRBTEST.COM (short username oryx)
I20260619 02:51:01.309495 25681 webserver.cc:460] Webserver started at http://127.18.134.60:38283/ using document root <none> and password file <none>
I20260619 02:51:01.309778 25681 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:01.309911 25681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:01.312204 25681 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 02:51:01.315092 25887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:01.315559 25681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260619 02:51:01.315675 25681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-2/wal
uuid: "59aa8b8f204d41a9b88df69b6eca76d3"
format_stamp: "Formatted at 2026-06-19 02:51:00 on dist-test-slave-9mr0"
server_key: "81d3e99cf8eb65b27d228c4d5f9b2681"
server_key_iv: "a7dd98181e8dc2886fa6cd1cc48f8d27"
server_key_version: "encryptionkey@0"
I20260619 02:51:01.315845 25681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:01.349102 25681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:01.352844 25681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:01.372475 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 25181
I20260619 02:51:01.389211 25681 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.60:44329
I20260619 02:51:01.389236 25938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.60:44329 every 8 connection(s)
I20260619 02:51:01.390321 25681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/master-2/data/info.pb
I20260619 02:51:01.394609 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:45679
--local_ip_for_outbound_sockets=127.18.134.1
--tserver_master_addrs=127.18.134.62:33075
--webserver_port=35633
--webserver_interface=127.18.134.1
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.1.keytab
--principal=oryx/127.18.134.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:35501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.134.62:33075,127.18.134.61:35581
--tserver_master_addrs=127.18.134.62:33075,127.18.134.61:35581,127.18.134.60:44329 with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:51:01.395741 25939 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:51:01.399165 25939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3: Bootstrap starting.
I20260619 02:51:01.414654 25939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:51:01.415356 25939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3: Bootstrap complete.
I20260619 02:51:01.416349 25939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: NON_VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: true } }
I20260619 02:51:01.416618 25939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 59aa8b8f204d41a9b88df69b6eca76d3, State: Initialized, Role: LEARNER
I20260619 02:51:01.416860 25939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: NON_VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: true } }
I20260619 02:51:01.417641 25943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: NON_VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: true } } }
I20260619 02:51:01.417966 25939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:51:01.418071 25943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [sys.catalog]: This master's current role is: LEARNER
I20260619 02:51:01.432591 25954 catalog_manager.cc:1259] Loaded cluster ID: 9cbe8949e88b4785a8e697bf8dceef26
I20260619 02:51:01.432690 25954 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3: loading cluster ID for follower catalog manager: success
I20260619 02:51:01.435189 25954 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3: acquiring CA information for follower catalog manager: success
I20260619 02:51:01.436937 25954 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 02:51:01.800539 25941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:01.800845 25941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:01.800983 25941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:01.808187 25941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:01.808303 25941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:01.808451 25941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:51:01.819428 25941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:35501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.1.keytab
--principal=oryx/127.18.134.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:45679
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=35633
--tserver_master_addrs=127.18.134.62:33075,127.18.134.61:35581,127.18.134.60:44329
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:01.820401 25959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.814270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44329 (local address 127.0.0.1:47640)
0619 02:51:01.814778 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:01.814800 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:01.814938 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:01.815333 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:01.815338 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:01.815426 (+ 88us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:01.815595 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:01.815603 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.816925 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.816930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:01.818337 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:01.818349 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.819654 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.819660 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:01.819824 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.819857 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 02:51:01.819944 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 02:51:01.820080 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":211,"threads_started":1}
I20260619 02:51:01.820415 25960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.814435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47640 (local address 127.18.134.60:44329)
0619 02:51:01.815033 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:01.815038 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:01.815060 (+ 22us) server_negotiation.cc:407] Connection header received
0619 02:51:01.815123 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:01.815128 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:01.815195 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:01.815294 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:01.815740 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.816802 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.818471 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.819482 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.819620 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.819852 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:01.819965 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 02:51:01.820089 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":362,"threads_started":1}
I20260619 02:51:01.820887 25941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:01.821256 25914 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260619 02:51:01.821800 25943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0094edcfb4284971a6d45b1ec4e48a30. Latest consensus state: current_term: 1 leader_uuid: "0094edcfb4284971a6d45b1ec4e48a30" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: NON_VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: true } } }
I20260619 02:51:01.822059 25943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [sys.catalog]: This master's current role is: LEARNER
I20260619 02:51:01.822435 25941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:01.835922 25962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:01.835832 25965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:01.835959 25963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:01.836532 25941 server_base.cc:1034] running on GCE node
I20260619 02:51:01.837126 25941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:01.838068 25941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:01.839303 25941 hybrid_clock.cc:648] HybridClock initialized: now 1781837461839228 us; error 50 us; skew 500 ppm
I20260619 02:51:01.839650 25941 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:51:01 dist-test-slave-9mr0 krb5kdc[25075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837461, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:51:01.842701 25941 init.cc:377] Logged in from keytab as oryx/127.18.134.1@KRBTEST.COM (short username oryx)
I20260619 02:51:01.844188 25941 webserver.cc:460] Webserver started at http://127.18.134.1:35633/ using document root <none> and password file <none>
I20260619 02:51:01.844877 25941 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:01.845002 25941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:01.850822 25941 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260619 02:51:01.853571 25972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:01.855098 25941 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:51:01.855247 25941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "10455eb59a7040a98cb6dc8988bc24b8"
format_stamp: "Formatted at 2026-06-19 02:50:58 on dist-test-slave-9mr0"
server_key: "bac4f025e19ee134097c8a184a94f407"
server_key_iv: "46ad1e3f531f44b366eec545b6c2db3b"
server_key_version: "encryptionkey@0"
I20260619 02:51:01.856210 25941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:01.869472 25941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:01.873461 25941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:01.873708 25941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:01.875113 25941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:51:01.876398 25941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:51:01.876482 25941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:01.876577 25941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:51:01.876645 25941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:01.909514 25941 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:45679
I20260619 02:51:01.909575 26084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:45679 every 8 connection(s)
I20260619 02:51:01.911055 25941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:51:01.921320 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 25941
I20260619 02:51:01.921666 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 25315
Jun 19 02:51:01 dist-test-slave-9mr0 krb5kdc[25075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837461, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.60@KRBTEST.COM
Jun 19 02:51:01 dist-test-slave-9mr0 krb5kdc[25075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837461, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.61@KRBTEST.COM
Jun 19 02:51:01 dist-test-slave-9mr0 krb5kdc[25075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837461, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM
I20260619 02:51:01.942169 26089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.913609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:54837 (local address 127.18.134.61:35581)
0619 02:51:01.914025 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:01.914030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:01.915309 (+ 1279us) server_negotiation.cc:407] Connection header received
0619 02:51:01.918072 (+ 2763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:01.918077 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:01.918159 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:01.918251 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:01.920970 (+ 2719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.922052 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.928341 (+ 6289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.929023 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.929122 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.934680 (+ 5558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:01.934689 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:01.934694 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:01.934733 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:01.938041 (+ 3308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.939703 (+ 1662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.939708 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.939712 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.939854 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.940302 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.940307 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.940310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.940619 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:01.940748 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:01.941573 (+ 825us) server_negotiation.cc:299] Negotiation successful
0619 02:51:01.941795 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":220,"threads_started":1}
I20260619 02:51:01.943612 26091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.914872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44329 (local address 127.18.134.1:34841)
0619 02:51:01.915484 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:01.915499 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:01.917722 (+ 2223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:01.918681 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:01.918700 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:01.919686 (+ 986us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:01.920760 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:01.920786 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.922258 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.922264 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:01.924635 (+ 2371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:01.924653 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.926052 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.926059 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:01.926434 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.928499 (+ 2065us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:01.928535 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:01.933469 (+ 4934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:01.937966 (+ 4497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:01.937977 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:01.937996 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:01.939274 (+ 1278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:01.939744 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:01.939748 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:01.939752 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:01.939990 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:01.940692 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:01.940700 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:01.941144 (+ 444us) client_negotiation.cc:769] Sending connection context
0619 02:51:01.941448 (+ 304us) client_negotiation.cc:240] Negotiation successful
0619 02:51:01.941987 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"mutex_wait_us":168,"spinlock_wait_cycles":96640,"thread_start_us":891,"threads_started":1}
I20260619 02:51:01.943641 26090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.914136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:35581 (local address 127.18.134.1:54837)
0619 02:51:01.915116 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:01.915169 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:01.917713 (+ 2544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:01.918681 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:01.918694 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:01.919685 (+ 991us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:01.920760 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:01.920786 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.922258 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.922264 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:01.928115 (+ 5851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:01.928133 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.929460 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.929467 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:01.929655 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.930310 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:01.930333 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:01.934397 (+ 4064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:01.938217 (+ 3820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:01.938222 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:01.938226 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:01.939573 (+ 1347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:01.939987 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:01.939991 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:01.939994 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:01.940163 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:01.940825 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:01.940829 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:01.941147 (+ 318us) client_negotiation.cc:769] Sending connection context
0619 02:51:01.942458 (+ 1311us) client_negotiation.cc:240] Negotiation successful
0619 02:51:01.942766 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"mutex_wait_us":141,"spinlock_wait_cycles":9088,"thread_start_us":849,"threads_started":1}
I20260619 02:51:01.944841 26092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.916439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.18.134.1:52631)
0619 02:51:01.916968 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:01.916985 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:01.917723 (+ 738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:01.928074 (+ 10351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:01.928079 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:01.928667 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:01.929036 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:01.929048 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.930826 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.930832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:01.932789 (+ 1957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:01.932804 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.933803 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.933809 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:01.933968 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.934731 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:01.934753 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:01.937735 (+ 2982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:01.941446 (+ 3711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:01.941450 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:01.941453 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:01.941957 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:01.942666 (+ 709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:01.942671 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:01.942674 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:01.942839 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:01.943434 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:01.943440 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:01.943608 (+ 168us) client_negotiation.cc:769] Sending connection context
0619 02:51:01.944204 (+ 596us) client_negotiation.cc:240] Negotiation successful
0619 02:51:01.944463 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":266,"threads_started":1}
I20260619 02:51:01.944883 25960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.914860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:34841 (local address 127.18.134.60:44329)
0619 02:51:01.915034 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:01.915041 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:01.915564 (+ 523us) server_negotiation.cc:407] Connection header received
0619 02:51:01.918118 (+ 2554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:01.918123 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:01.918191 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:01.918271 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:01.920969 (+ 2698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.922068 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.924854 (+ 2786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.925554 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.925664 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.933731 (+ 8067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:01.933736 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:01.933740 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:01.933773 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:01.937777 (+ 4004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.939437 (+ 1660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.939443 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.939447 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.939599 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.940124 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.940128 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.940132 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.940487 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:01.940622 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:01.944316 (+ 3694us) server_negotiation.cc:299] Negotiation successful
0619 02:51:01.944562 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Jun 19 02:51:01 dist-test-slave-9mr0 krb5kdc[25075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.134.2@KRBTEST.COM
I20260619 02:51:01.945545 26087 heartbeater.cc:347] Connected to a master server at 127.18.134.61:35581
I20260619 02:51:01.945619 26085 heartbeater.cc:347] Connected to a master server at 127.18.134.60:44329
I20260619 02:51:01.945873 26085 heartbeater.cc:464] Registering TS with master...
I20260619 02:51:01.945876 26087 heartbeater.cc:464] Registering TS with master...
I20260619 02:51:01.946218 26093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.916595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:52631 (local address 127.18.134.62:33075)
0619 02:51:01.927778 (+ 11183us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:01.927784 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:01.927807 (+ 23us) server_negotiation.cc:407] Connection header received
0619 02:51:01.927859 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:01.927863 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:01.927922 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:01.928011 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:01.929227 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.930623 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.932980 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.933592 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.934239 (+ 647us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.937995 (+ 3756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:01.938001 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:01.938006 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:01.938044 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:01.941305 (+ 3261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.942102 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.942106 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.942109 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.942507 (+ 398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.942974 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.942979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.942982 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.943307 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:01.945472 (+ 2165us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:01.945742 (+ 270us) server_negotiation.cc:299] Negotiation successful
0619 02:51:01.945914 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11032,"thread_start_us":11031,"threads_started":1}
I20260619 02:51:01.946863 26086 heartbeater.cc:347] Connected to a master server at 127.18.134.62:33075
I20260619 02:51:01.946947 26085 heartbeater.cc:511] Master 127.18.134.60:44329 requested a full tablet report, sending...
I20260619 02:51:01.946995 26086 heartbeater.cc:464] Registering TS with master...
I20260619 02:51:01.946970 26087 heartbeater.cc:511] Master 127.18.134.61:35581 requested a full tablet report, sending...
I20260619 02:51:01.947422 26086 heartbeater.cc:511] Master 127.18.134.62:33075 requested a full tablet report, sending...
I20260619 02:51:01.948967 25119 ts_manager.cc:194] Re-registered known tserver with Master: 10455eb59a7040a98cb6dc8988bc24b8 (127.18.134.1:45679)
I20260619 02:51:01.949306 25717 ts_manager.cc:194] Registered new tserver with Master: 10455eb59a7040a98cb6dc8988bc24b8 (127.18.134.1:45679)
I20260619 02:51:01.949767 25904 ts_manager.cc:194] Registered new tserver with Master: 10455eb59a7040a98cb6dc8988bc24b8 (127.18.134.1:45679)
I20260619 02:51:01.950055 25119 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.134.1@KRBTEST.COM'} at 127.18.134.1:52631
I20260619 02:51:01.954595 26095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:01.923037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38176 (local address 127.18.134.2:45195)
0619 02:51:01.933734 (+ 10697us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:01.933743 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:01.933774 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:51:01.933837 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:01.933847 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:01.934138 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:01.934301 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:01.936425 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.938192 (+ 1767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.939972 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:01.940834 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:01.940954 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:01.948752 (+ 7798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:01.948769 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:01.948786 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:01.948836 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:01.951895 (+ 3059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.952641 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.952650 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.952654 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.952786 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:01.953128 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:01.953134 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:01.953138 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:01.953449 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:01.953731 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:01.954004 (+ 273us) server_negotiation.cc:299] Negotiation successful
0619 02:51:01.954262 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10517,"thread_start_us":10366,"threads_started":1}
I20260619 02:51:01.955732 25438 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:02.036343 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 25315
I20260619 02:51:02.048810 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:45195
--local_ip_for_outbound_sockets=127.18.134.2
--tserver_master_addrs=127.18.134.62:33075
--webserver_port=38569
--webserver_interface=127.18.134.2
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.2.keytab
--principal=oryx/127.18.134.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:35501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.134.62:33075,127.18.134.61:35581
--tserver_master_addrs=127.18.134.62:33075,127.18.134.61:35581,127.18.134.60:44329 with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:51:02.369755 26097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:02.370045 26097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:02.370178 26097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:02.378001 26097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:02.378108 26097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:02.378261 26097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:51:02.388477 26097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:35501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.2.keytab
--principal=oryx/127.18.134.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:45195
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=38569
--tserver_master_addrs=127.18.134.62:33075,127.18.134.61:35581,127.18.134.60:44329
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:02.390062 26097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:02.391690 26097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:51:02.400233 26101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0094edcfb4284971a6d45b1ec4e48a30" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: NON_VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: true } } }
I20260619 02:51:02.401290 26101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [sys.catalog]: This master's current role is: LEADER
I20260619 02:51:02.407444 26103 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30: attempting to promote NON_VOTER 59aa8b8f204d41a9b88df69b6eca76d3 to VOTER
I20260619 02:51:02.408365 26103 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: false } }
I20260619 02:51:02.410617 25727 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [term 1 FOLLOWER]: Refusing update from remote peer 0094edcfb4284971a6d45b1ec4e48a30: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260619 02:51:02.410712 25914 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [term 1 LEARNER]: Refusing update from remote peer 0094edcfb4284971a6d45b1ec4e48a30: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260619 02:51:02.411540 26103 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260619 02:51:02.411996 26101 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260619 02:51:02.412700 26108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:02.413468 26107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:02.414052 26097 server_base.cc:1034] running on GCE node
W20260619 02:51:02.414746 26110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:02.414839 26101 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 59aa8b8f204d41a9b88df69b6eca76d3 (127.18.134.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: false } } }
I20260619 02:51:02.415529 26097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:02.415593 25727 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 59aa8b8f204d41a9b88df69b6eca76d3 (127.18.134.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: false } } }
I20260619 02:51:02.416870 26097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:02.418419 26097 hybrid_clock.cc:648] HybridClock initialized: now 1781837462418343 us; error 118 us; skew 500 ppm
I20260619 02:51:02.418843 26097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:51:02.418764 26102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0094edcfb4284971a6d45b1ec4e48a30" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: false } } }
I20260619 02:51:02.418845 26104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0094edcfb4284971a6d45b1ec4e48a30" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: false } } }
I20260619 02:51:02.419225 26104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [sys.catalog]: This master's current role is: LEADER
I20260619 02:51:02.419127 26102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [sys.catalog]: This master's current role is: LEADER
I20260619 02:51:02.418159 26101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0094edcfb4284971a6d45b1ec4e48a30" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: false } } }
I20260619 02:51:02.420091 26101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30 [sys.catalog]: This master's current role is: LEADER
I20260619 02:51:02.420634 26116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0094edcfb4284971a6d45b1ec4e48a30" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: false } } }
I20260619 02:51:02.420203 25914 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 59aa8b8f204d41a9b88df69b6eca76d3 (127.18.134.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: false } } }
I20260619 02:51:02.421018 26116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [sys.catalog]: This master's current role is: FOLLOWER
Jun 19 02:51:02 dist-test-slave-9mr0 krb5kdc[25075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837462, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:51:02.423242 26097 init.cc:377] Logged in from keytab as oryx/127.18.134.2@KRBTEST.COM (short username oryx)
I20260619 02:51:02.424427 26119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0094edcfb4284971a6d45b1ec4e48a30" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: false } } }
I20260619 02:51:02.424815 26119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:51:02.425676 26097 webserver.cc:460] Webserver started at http://127.18.134.2:38569/ using document root <none> and password file <none>
I20260619 02:51:02.426360 26097 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:02.426476 26097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:02.431874 26097 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.001s
I20260619 02:51:02.434463 26122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:02.435834 26097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:51:02.435977 26097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "f46cfab95e6e44c0be837c69674063c9"
format_stamp: "Formatted at 2026-06-19 02:50:59 on dist-test-slave-9mr0"
server_key: "598037b8ebb881b36a0a508f9072ec38"
server_key_iv: "0a42beded8afe2e32fedb28aa286b75f"
server_key_version: "encryptionkey@0"
I20260619 02:51:02.436946 26097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:02.458892 26097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:02.463505 26097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:02.463733 26097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:02.464697 26097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:51:02.465989 26097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:51:02.466070 26097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:02.466143 26097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:51:02.466194 26097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:02.493870 26097 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:45195
I20260619 02:51:02.493917 26234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:45195 every 8 connection(s)
I20260619 02:51:02.495255 26097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:51:02.498421 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 26097
I20260619 02:51:02.498714 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 25448
Jun 19 02:51:02 dist-test-slave-9mr0 krb5kdc[25075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837462, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.61@KRBTEST.COM
Jun 19 02:51:02 dist-test-slave-9mr0 krb5kdc[25075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837462, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.60@KRBTEST.COM
Jun 19 02:51:02 dist-test-slave-9mr0 krb5kdc[25075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.134.3@KRBTEST.COM
I20260619 02:51:02.524526 26239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:02.497961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:32947 (local address 127.18.134.60:44329)
0619 02:51:02.498414 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:02.498418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:02.505695 (+ 7277us) server_negotiation.cc:407] Connection header received
0619 02:51:02.505761 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:02.505766 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:02.505897 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:02.506002 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:02.507465 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:02.508779 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:02.511043 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:02.511658 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:02.511778 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:02.516698 (+ 4920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:02.516705 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:02.516710 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:02.516749 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:02.520398 (+ 3649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:02.521876 (+ 1478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:02.521882 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:02.521886 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:02.522030 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:02.522500 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:02.522505 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:02.522507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:02.522878 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:02.523012 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:02.523951 (+ 939us) server_negotiation.cc:299] Negotiation successful
0619 02:51:02.524163 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":233,"threads_started":1}
I20260619 02:51:02.525209 26241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:02.499234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:55321 (local address 127.18.134.61:35581)
0619 02:51:02.503504 (+ 4270us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:02.503510 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:02.503530 (+ 20us) server_negotiation.cc:407] Connection header received
0619 02:51:02.503597 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:02.503603 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:02.503699 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:02.503826 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:02.506145 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:02.507360 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:02.510171 (+ 2811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:02.510849 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:02.510946 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:02.516691 (+ 5745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:02.516698 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:02.516702 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:02.516753 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:02.520512 (+ 3759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:02.522064 (+ 1552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:02.522070 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:02.522073 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:02.522242 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:02.522720 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:02.522725 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:02.522728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:02.523043 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:02.523158 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:02.524650 (+ 1492us) server_negotiation.cc:299] Negotiation successful
0619 02:51:02.524862 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4143,"thread_start_us":4046,"threads_started":1}
I20260619 02:51:02.525700 26242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:02.499322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44329 (local address 127.18.134.2:32947)
0619 02:51:02.505476 (+ 6154us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:02.505492 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:02.505591 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:02.506218 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:02.506270 (+ 52us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:02.506958 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:02.507279 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:02.507290 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:02.508953 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:02.508958 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:02.510850 (+ 1892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:02.510866 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:02.512037 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:02.512043 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:02.512214 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:02.513307 (+ 1093us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:02.513329 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:02.516435 (+ 3106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:02.520549 (+ 4114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:02.520559 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:02.520581 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:02.521714 (+ 1133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:02.522153 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:02.522158 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:02.522162 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:02.522387 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:02.523085 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:02.523094 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:02.523549 (+ 455us) client_negotiation.cc:769] Sending connection context
0619 02:51:02.523820 (+ 271us) client_negotiation.cc:240] Negotiation successful
0619 02:51:02.524327 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6025,"mutex_wait_us":3,"thread_start_us":5840,"threads_started":1}
I20260619 02:51:02.525729 26240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:02.498601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:35581 (local address 127.18.134.2:55321)
0619 02:51:02.499549 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:02.499610 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:02.502782 (+ 3172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:02.504138 (+ 1356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:02.504150 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:02.505007 (+ 857us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:02.505930 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:02.505956 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:02.507572 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:02.507577 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:02.509989 (+ 2412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:02.510004 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:02.511317 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:02.511324 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:02.511617 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:02.512434 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:02.512466 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:02.516435 (+ 3969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:02.520688 (+ 4253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:02.520693 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:02.520697 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:02.521919 (+ 1222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:02.522384 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:02.522388 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:02.522392 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:02.522559 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:02.523257 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:02.523261 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:02.524357 (+ 1096us) client_negotiation.cc:769] Sending connection context
0619 02:51:02.525044 (+ 687us) client_negotiation.cc:240] Negotiation successful
0619 02:51:02.525327 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"mutex_wait_us":836,"thread_start_us":668,"threads_started":1}
I20260619 02:51:02.527415 26235 heartbeater.cc:347] Connected to a master server at 127.18.134.60:44329
I20260619 02:51:02.527480 26237 heartbeater.cc:347] Connected to a master server at 127.18.134.61:35581
I20260619 02:51:02.527762 26235 heartbeater.cc:464] Registering TS with master...
I20260619 02:51:02.527777 26237 heartbeater.cc:464] Registering TS with master...
Jun 19 02:51:02 dist-test-slave-9mr0 krb5kdc[25075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837462, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM
I20260619 02:51:02.528914 26237 heartbeater.cc:511] Master 127.18.134.61:35581 requested a full tablet report, sending...
I20260619 02:51:02.528940 26235 heartbeater.cc:511] Master 127.18.134.60:44329 requested a full tablet report, sending...
I20260619 02:51:02.530608 25904 ts_manager.cc:194] Registered new tserver with Master: f46cfab95e6e44c0be837c69674063c9 (127.18.134.2:45195)
I20260619 02:51:02.530812 25717 ts_manager.cc:194] Registered new tserver with Master: f46cfab95e6e44c0be837c69674063c9 (127.18.134.2:45195)
I20260619 02:51:02.532074 26244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:02.499997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55594 (local address 127.18.134.3:34939)
0619 02:51:02.504416 (+ 4419us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:02.504425 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:02.504895 (+ 470us) server_negotiation.cc:407] Connection header received
0619 02:51:02.505072 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:02.505081 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:02.505330 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:02.505476 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:02.512954 (+ 7478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:02.514467 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:02.516419 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:02.517266 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:02.521501 (+ 4235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:02.523283 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:02.523298 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:02.523317 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:02.523360 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:02.527008 (+ 3648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:02.528513 (+ 1505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:02.528522 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:02.528526 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:02.528684 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:02.529112 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:02.529117 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:02.529120 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:02.530234 (+ 1114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:02.530514 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:02.531606 (+ 1092us) server_negotiation.cc:299] Negotiation successful
0619 02:51:02.531802 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4247,"thread_start_us":4090,"threads_started":1}
I20260619 02:51:02.533390 25571 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:02.539374 26246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:02.505771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:43867 (local address 127.18.134.62:33075)
0619 02:51:02.514965 (+ 9194us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:02.514996 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:02.517528 (+ 2532us) server_negotiation.cc:407] Connection header received
0619 02:51:02.517595 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:02.517600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:02.517693 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:02.517797 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:02.519027 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:02.520223 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:02.522507 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:02.523176 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:02.523293 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:02.529322 (+ 6029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:02.529327 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:02.529330 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:02.529369 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:02.532974 (+ 3605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:02.537479 (+ 4505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:02.537483 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:02.537485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:02.537626 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:02.538009 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:02.538012 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:02.538014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:02.538274 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:02.538401 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:02.538804 (+ 403us) server_negotiation.cc:299] Negotiation successful
0619 02:51:02.539007 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9021,"thread_start_us":8807,"threads_started":1}
I20260619 02:51:02.539372 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:02.505644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.18.134.2:43867)
0619 02:51:02.517132 (+ 11488us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:02.517150 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:02.517283 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:02.517988 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:02.517993 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:02.518486 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:02.518863 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:02.518874 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:02.520413 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:02.520418 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:02.522339 (+ 1921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:02.522352 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:02.523524 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:02.523530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:02.523718 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:02.525906 (+ 2188us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:02.526391 (+ 485us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:02.529107 (+ 2716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:02.536910 (+ 7803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:02.536917 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:02.536921 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:02.537334 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:02.537741 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:02.537747 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:02.537752 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:02.537896 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:02.538380 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:02.538385 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:02.538519 (+ 134us) client_negotiation.cc:769] Sending connection context
0619 02:51:02.538647 (+ 128us) client_negotiation.cc:240] Negotiation successful
0619 02:51:02.538940 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11149,"thread_start_us":11241,"threads_started":1}
I20260619 02:51:02.540140 26236 heartbeater.cc:347] Connected to a master server at 127.18.134.62:33075
I20260619 02:51:02.540331 26236 heartbeater.cc:464] Registering TS with master...
I20260619 02:51:02.540838 26236 heartbeater.cc:511] Master 127.18.134.62:33075 requested a full tablet report, sending...
I20260619 02:51:02.541829 25119 ts_manager.cc:194] Re-registered known tserver with Master: f46cfab95e6e44c0be837c69674063c9 (127.18.134.2:45195)
I20260619 02:51:02.542662 25119 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.134.2@KRBTEST.COM'} at 127.18.134.2:43867
I20260619 02:51:02.615885 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 25448
I20260619 02:51:02.628562 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:34939
--local_ip_for_outbound_sockets=127.18.134.3
--tserver_master_addrs=127.18.134.62:33075
--webserver_port=36349
--webserver_interface=127.18.134.3
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.3.keytab
--principal=oryx/127.18.134.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:35501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.134.62:33075,127.18.134.61:35581
--tserver_master_addrs=127.18.134.62:33075,127.18.134.61:35581,127.18.134.60:44329 with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:51:02.943961 26248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:02.944235 26248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:02.944357 26248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:02.951835 26248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:02.951989 26248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:02.952226 26248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:51:02.952234 26086 heartbeater.cc:503] Master 127.18.134.62:33075 was elected leader, sending a full tablet report...
I20260619 02:51:02.962247 26248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:35501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.3.keytab
--principal=oryx/127.18.134.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:34939
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=36349
--tserver_master_addrs=127.18.134.62:33075,127.18.134.61:35581,127.18.134.60:44329
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:02.963500 26248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:02.964737 26248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:02.977043 26253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:02.976994 26256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:02.977116 26254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:02.978142 26248 server_base.cc:1034] running on GCE node
I20260619 02:51:02.978789 26248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:02.979708 26248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:02.980921 26248 hybrid_clock.cc:648] HybridClock initialized: now 1781837462980857 us; error 56 us; skew 500 ppm
I20260619 02:51:02.981230 26248 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:51:02 dist-test-slave-9mr0 krb5kdc[25075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837462, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:51:02.984337 26248 init.cc:377] Logged in from keytab as oryx/127.18.134.3@KRBTEST.COM (short username oryx)
I20260619 02:51:02.985769 26248 webserver.cc:460] Webserver started at http://127.18.134.3:36349/ using document root <none> and password file <none>
I20260619 02:51:02.986441 26248 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:02.986577 26248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:02.992018 26248 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:51:02.994733 26263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:02.996104 26248 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 02:51:02.996269 26248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "7c852f00331741bf97c3d710d873c775"
format_stamp: "Formatted at 2026-06-19 02:50:59 on dist-test-slave-9mr0"
server_key: "c7b20881c45f2973ea33ca58b71ff7f8"
server_key_iv: "5223e859469b4f6305982b1ed4fd6fd3"
server_key_version: "encryptionkey@0"
I20260619 02:51:02.997208 26248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:03.010473 26248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:03.014593 26248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:03.014962 26248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:03.016276 26248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:51:03.017638 26248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:51:03.017730 26248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:03.017833 26248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:51:03.017903 26248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:03.046840 26248 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:34939
I20260619 02:51:03.046907 26375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:34939 every 8 connection(s)
I20260619 02:51:03.048245 26248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:51:03.057245 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 26248
Jun 19 02:51:03 dist-test-slave-9mr0 krb5kdc[25075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837462, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM
Jun 19 02:51:03 dist-test-slave-9mr0 krb5kdc[25075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837462, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.60@KRBTEST.COM
I20260619 02:51:03.113487 26391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:03.076013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50526 (local address 127.18.134.61:35581)
0619 02:51:03.092127 (+ 16114us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:03.092134 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:03.092152 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:51:03.092212 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:03.092217 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:03.092302 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:03.092412 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:03.096585 (+ 4173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.097720 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.099646 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.100445 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.100554 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:03.104461 (+ 3907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:03.104467 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:03.104472 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:03.104510 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:03.109128 (+ 4618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:03.110432 (+ 1304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:03.110439 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:03.110443 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:03.110680 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:03.111175 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:03.111180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:03.111183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:03.111513 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:03.111770 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:03.112875 (+ 1105us) server_negotiation.cc:299] Negotiation successful
0619 02:51:03.113094 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":81,"server-negotiator.queue_time_us":15982,"thread_start_us":16516,"threads_started":1}
I20260619 02:51:03.118047 26385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:03.053325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:41485 (local address 127.18.134.62:33075)
0619 02:51:03.073713 (+ 20388us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:03.073720 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:03.096164 (+ 22444us) server_negotiation.cc:407] Connection header received
0619 02:51:03.096291 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:03.096296 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:03.096371 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:03.096466 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:03.097681 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.098739 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.101320 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.101929 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.102025 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:03.108662 (+ 6637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:03.108669 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:03.108675 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:03.108715 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:03.112140 (+ 3425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:03.114778 (+ 2638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:03.114783 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:03.114787 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:03.114945 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:03.115498 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:03.115504 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:03.115507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:03.115861 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:03.115986 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:03.117364 (+ 1378us) server_negotiation.cc:299] Negotiation successful
0619 02:51:03.117619 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20230,"thread_start_us":20394,"threads_started":1}
Jun 19 02:51:03 dist-test-slave-9mr0 krb5kdc[25075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837462, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.61@KRBTEST.COM
I20260619 02:51:03.120590 26383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:03.053171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.18.134.3:41485)
0619 02:51:03.096043 (+ 42872us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:03.096059 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:03.096173 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:03.096627 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:03.096631 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:03.097177 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:03.097512 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:03.097524 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.098939 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.098944 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:03.101143 (+ 2199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:03.101157 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.102388 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.102395 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:03.103236 (+ 841us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:03.103985 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:03.104026 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:03.107492 (+ 3466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:03.113889 (+ 6397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:03.113899 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:03.113921 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:03.114589 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:03.115100 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:03.115105 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:03.115108 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:03.115369 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:03.116101 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:03.116112 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:03.116863 (+ 751us) client_negotiation.cc:769] Sending connection context
0619 02:51:03.118407 (+ 1544us) client_negotiation.cc:240] Negotiation successful
0619 02:51:03.119041 (+ 634us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42755,"mutex_wait_us":1,"thread_start_us":39807,"threads_started":1}
I20260619 02:51:03.122242 26377 heartbeater.cc:347] Connected to a master server at 127.18.134.62:33075
I20260619 02:51:03.122555 26377 heartbeater.cc:464] Registering TS with master...
I20260619 02:51:03.123524 26377 heartbeater.cc:511] Master 127.18.134.62:33075 requested a full tablet report, sending...
I20260619 02:51:03.124564 26382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:03.051633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44329 (local address 127.18.134.3:35225)
0619 02:51:03.052534 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:03.052595 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:03.067943 (+ 15348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:03.094486 (+ 26543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:03.094502 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:03.095429 (+ 927us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:03.096409 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:03.096435 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.101564 (+ 5129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.101570 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:03.103561 (+ 1991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:03.103576 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.105067 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.105073 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:03.106299 (+ 1226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:03.106935 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:03.106961 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:03.112867 (+ 5906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:03.121459 (+ 8592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:03.121464 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:03.121468 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:03.121850 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:03.122621 (+ 771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:03.122671 (+ 50us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:03.122674 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:03.122857 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:03.123554 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:03.123559 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:03.123724 (+ 165us) client_negotiation.cc:769] Sending connection context
0619 02:51:03.123858 (+ 134us) client_negotiation.cc:240] Negotiation successful
0619 02:51:03.124150 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":358,"threads_started":1}
I20260619 02:51:03.125582 25119 ts_manager.cc:194] Re-registered known tserver with Master: 7c852f00331741bf97c3d710d873c775 (127.18.134.3:34939)
I20260619 02:51:03.125923 26381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:03.051391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:35225 (local address 127.18.134.60:44329)
0619 02:51:03.093599 (+ 42208us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:03.093606 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:03.093624 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:51:03.093682 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:03.093688 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:03.093794 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:03.096338 (+ 2544us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:03.098228 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.099562 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.104039 (+ 4477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.104705 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.104821 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:03.113089 (+ 8268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:03.113096 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:03.113100 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:03.113139 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:03.121311 (+ 8172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:03.122325 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:03.122331 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:03.122334 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:03.122485 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:03.122997 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:03.123002 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:03.123005 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:03.123340 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:03.123477 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:03.124844 (+ 1367us) server_negotiation.cc:299] Negotiation successful
0619 02:51:03.125064 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42087,"thread_start_us":41841,"threads_started":1}
I20260619 02:51:03.126816 25119 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.134.3@KRBTEST.COM'} at 127.18.134.3:41485
I20260619 02:51:03.128152 26376 heartbeater.cc:347] Connected to a master server at 127.18.134.60:44329
I20260619 02:51:03.128309 26376 heartbeater.cc:464] Registering TS with master...
I20260619 02:51:03.128505 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:03.051114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:59929 (local address 127.18.134.61:35581)
0619 02:51:03.051545 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:03.051552 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:03.093406 (+ 41854us) server_negotiation.cc:407] Connection header received
0619 02:51:03.096588 (+ 3182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:03.096594 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:03.096683 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:03.096805 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:03.099774 (+ 2969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.101114 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.113379 (+ 12265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.114044 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.114145 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:03.121237 (+ 7092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:03.121244 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:03.121248 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:03.121297 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:03.124558 (+ 3261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:03.125326 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:03.125332 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:03.125336 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:03.125858 (+ 522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:03.126896 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:03.126902 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:03.126905 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:03.127242 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:03.127370 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:03.127896 (+ 526us) server_negotiation.cc:299] Negotiation successful
0619 02:51:03.128105 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":310,"thread_start_us":358,"threads_started":1}
I20260619 02:51:03.128715 26376 heartbeater.cc:511] Master 127.18.134.60:44329 requested a full tablet report, sending...
I20260619 02:51:03.129225 26394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:03.092170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47654 (local address 127.18.134.60:44329)
0619 02:51:03.105811 (+ 13641us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:03.105815 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:03.106435 (+ 620us) server_negotiation.cc:407] Connection header received
0619 02:51:03.106488 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:03.106492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:03.106563 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:03.106701 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:03.108802 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.110483 (+ 1681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.112361 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.113018 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.113174 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:03.116656 (+ 3482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:03.116667 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:03.116673 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:03.116712 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:03.120536 (+ 3824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:03.126328 (+ 5792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:03.126333 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:03.126336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:03.126508 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:03.127027 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:03.127031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:03.127034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:03.127340 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:03.127549 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:03.128675 (+ 1126us) server_negotiation.cc:299] Negotiation successful
0619 02:51:03.128887 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13530,"thread_start_us":13416,"threads_started":1}
I20260619 02:51:03.129412 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:03.051633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:35581 (local address 127.18.134.3:59929)
0619 02:51:03.093295 (+ 41662us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:03.093317 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:03.094009 (+ 692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:03.097105 (+ 3096us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:03.097110 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:03.097706 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:03.098020 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:03.098032 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.109744 (+ 11712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.109749 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:03.112188 (+ 2439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:03.112205 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.115987 (+ 3782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.115993 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:03.117155 (+ 1162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:03.117675 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:03.117696 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:03.120953 (+ 3257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:03.124712 (+ 3759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:03.124717 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:03.124721 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:03.125185 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:03.126565 (+ 1380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:03.126573 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:03.126577 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:03.126766 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:03.127457 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:03.127461 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:03.127614 (+ 153us) client_negotiation.cc:769] Sending connection context
0619 02:51:03.128726 (+ 1112us) client_negotiation.cc:240] Negotiation successful
0619 02:51:03.129011 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41498,"mutex_wait_us":960,"spinlock_wait_cycles":63872,"thread_start_us":43980,"threads_started":1}
I20260619 02:51:03.130350 25904 ts_manager.cc:194] Registered new tserver with Master: 7c852f00331741bf97c3d710d873c775 (127.18.134.3:34939)
I20260619 02:51:03.130617 26378 heartbeater.cc:347] Connected to a master server at 127.18.134.61:35581
I20260619 02:51:03.130880 26378 heartbeater.cc:464] Registering TS with master...
I20260619 02:51:03.131548 26378 heartbeater.cc:511] Master 127.18.134.61:35581 requested a full tablet report, sending...
I20260619 02:51:03.132280 26390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:03.076011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37792 (local address 127.18.134.62:33075)
0619 02:51:03.106096 (+ 30085us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:03.106102 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:03.106124 (+ 22us) server_negotiation.cc:407] Connection header received
0619 02:51:03.106177 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:03.106181 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:03.106260 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:03.106365 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:03.112371 (+ 6006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.117001 (+ 4630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.118902 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.119508 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.119641 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:03.123936 (+ 4295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:03.123943 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:03.123948 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:03.123988 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:03.127323 (+ 3335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:03.128123 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:03.128128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:03.128131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:03.128257 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:03.128678 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:03.128683 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:03.128686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:03.128990 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:03.129196 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:03.131684 (+ 2488us) server_negotiation.cc:299] Negotiation successful
0619 02:51:03.131905 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29979,"thread_start_us":31264,"threads_started":1}
I20260619 02:51:03.132588 25717 ts_manager.cc:194] Registered new tserver with Master: 7c852f00331741bf97c3d710d873c775 (127.18.134.3:34939)
I20260619 02:51:03.136114 25119 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37792:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:51:03.143510 25119 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:51:03.173511 26406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:03.163225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:45195 (local address 127.0.0.1:38186)
0619 02:51:03.164238 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:03.164253 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:03.164347 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:03.165568 (+ 1221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:03.165573 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:03.165599 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:03.165925 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:03.165936 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.167906 (+ 1970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.167912 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:03.170097 (+ 2185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:03.170113 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.171982 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.171989 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:03.172176 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:03.172215 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 02:51:03.172305 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 02:51:03.172998 (+ 693us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":908,"mutex_wait_us":3,"thread_start_us":1504,"threads_started":1}
I20260619 02:51:03.174564 26408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:03.164374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38186 (local address 127.18.134.2:45195)
0619 02:51:03.164773 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:03.164782 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:03.164811 (+ 29us) server_negotiation.cc:407] Connection header received
0619 02:51:03.164918 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:03.164925 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:03.165221 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:03.165444 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:03.166078 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.167755 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.170338 (+ 2583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.171779 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.173831 (+ 2052us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:03.174045 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:03.174123 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 02:51:03.174305 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":192,"threads_started":1}
I20260619 02:51:03.179136 26403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:03.162394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:45679 (local address 127.0.0.1:49660)
0619 02:51:03.163049 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:03.163071 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:03.163215 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:03.173254 (+ 10039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:03.173259 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:03.173284 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:03.173642 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:03.173655 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.175197 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.175202 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:03.176851 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:03.176863 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.178411 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.178417 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:03.178590 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:03.178660 (+ 70us) client_negotiation.cc:769] Sending connection context
0619 02:51:03.178723 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 02:51:03.178864 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":334,"threads_started":1}
I20260619 02:51:03.178474 26170 tablet_service.cc:1467] Processing CreateTablet for tablet 9a9c3a545dff474284dfa07856c8db6e (DEFAULT_TABLE table=test-table [id=b0b5ad99a9f746279e54917d22795e1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:51:03.180025 26404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:03.162564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49660 (local address 127.18.134.1:45679)
0619 02:51:03.172693 (+ 10129us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:03.172701 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:03.172732 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:51:03.172805 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:03.172810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:03.173049 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:03.173847 (+ 798us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:03.174002 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.175063 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.176999 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.178187 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.179459 (+ 1272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:03.179622 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:03.179675 (+ 53us) server_negotiation.cc:299] Negotiation successful
0619 02:51:03.179795 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9982,"thread_start_us":9881,"threads_started":1}
I20260619 02:51:03.180094 26170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a9c3a545dff474284dfa07856c8db6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:03.180727 26405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:03.163216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:34939 (local address 127.0.0.1:55602)
0619 02:51:03.163720 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:03.163737 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:03.163830 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:03.172045 (+ 8215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:03.172049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:03.172077 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:03.172460 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:03.172472 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.176186 (+ 3714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.176191 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:03.178203 (+ 2012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:03.178227 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.180049 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.180055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:03.180225 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:03.180257 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 02:51:03.180352 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 02:51:03.180470 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":955,"threads_started":1}
I20260619 02:51:03.180861 26407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:03.163986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55602 (local address 127.18.134.3:34939)
0619 02:51:03.171270 (+ 7284us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:03.171281 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:03.171323 (+ 42us) server_negotiation.cc:407] Connection header received
0619 02:51:03.171444 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:03.171453 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:03.171792 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:03.172065 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:03.172674 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.176042 (+ 3368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.178403 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.179845 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.179972 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:03.180162 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:03.180402 (+ 240us) server_negotiation.cc:299] Negotiation successful
0619 02:51:03.180557 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7056,"thread_start_us":6862,"threads_started":1}
I20260619 02:51:03.184424 26311 tablet_service.cc:1467] Processing CreateTablet for tablet 9a9c3a545dff474284dfa07856c8db6e (DEFAULT_TABLE table=test-table [id=b0b5ad99a9f746279e54917d22795e1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:51:03.185093 26020 tablet_service.cc:1467] Processing CreateTablet for tablet 9a9c3a545dff474284dfa07856c8db6e (DEFAULT_TABLE table=test-table [id=b0b5ad99a9f746279e54917d22795e1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:51:03.186460 26020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a9c3a545dff474284dfa07856c8db6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:03.186460 26311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a9c3a545dff474284dfa07856c8db6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:03.193348 26409 tablet_bootstrap.cc:492] T 9a9c3a545dff474284dfa07856c8db6e P f46cfab95e6e44c0be837c69674063c9: Bootstrap starting.
I20260619 02:51:03.196925 26409 tablet_bootstrap.cc:654] T 9a9c3a545dff474284dfa07856c8db6e P f46cfab95e6e44c0be837c69674063c9: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:03.198215 26409 log.cc:824] T 9a9c3a545dff474284dfa07856c8db6e P f46cfab95e6e44c0be837c69674063c9: Log is configured to *not* fsync() on all Append() calls
I20260619 02:51:03.202091 26409 tablet_bootstrap.cc:492] T 9a9c3a545dff474284dfa07856c8db6e P f46cfab95e6e44c0be837c69674063c9: No bootstrap required, opened a new log
I20260619 02:51:03.202112 26412 tablet_bootstrap.cc:492] T 9a9c3a545dff474284dfa07856c8db6e P 7c852f00331741bf97c3d710d873c775: Bootstrap starting.
I20260619 02:51:03.202558 26409 ts_tablet_manager.cc:1397] T 9a9c3a545dff474284dfa07856c8db6e P f46cfab95e6e44c0be837c69674063c9: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20260619 02:51:03.203311 26411 tablet_bootstrap.cc:492] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8: Bootstrap starting.
I20260619 02:51:03.207520 26411 tablet_bootstrap.cc:654] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:03.208348 26412 tablet_bootstrap.cc:654] T 9a9c3a545dff474284dfa07856c8db6e P 7c852f00331741bf97c3d710d873c775: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:03.208792 26411 log.cc:824] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8: Log is configured to *not* fsync() on all Append() calls
I20260619 02:51:03.209462 26412 log.cc:824] T 9a9c3a545dff474284dfa07856c8db6e P 7c852f00331741bf97c3d710d873c775: Log is configured to *not* fsync() on all Append() calls
I20260619 02:51:03.212355 26411 tablet_bootstrap.cc:492] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8: No bootstrap required, opened a new log
I20260619 02:51:03.212667 26411 ts_tablet_manager.cc:1397] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260619 02:51:03.212944 26409 raft_consensus.cc:348] T 9a9c3a545dff474284dfa07856c8db6e P f46cfab95e6e44c0be837c69674063c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10455eb59a7040a98cb6dc8988bc24b8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45679 } } peers { permanent_uuid: "f46cfab95e6e44c0be837c69674063c9" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45195 } } peers { permanent_uuid: "7c852f00331741bf97c3d710d873c775" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34939 } }
I20260619 02:51:03.213311 26409 raft_consensus.cc:374] T 9a9c3a545dff474284dfa07856c8db6e P f46cfab95e6e44c0be837c69674063c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:03.213369 26412 tablet_bootstrap.cc:492] T 9a9c3a545dff474284dfa07856c8db6e P 7c852f00331741bf97c3d710d873c775: No bootstrap required, opened a new log
I20260619 02:51:03.213433 26409 raft_consensus.cc:729] T 9a9c3a545dff474284dfa07856c8db6e P f46cfab95e6e44c0be837c69674063c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46cfab95e6e44c0be837c69674063c9, State: Initialized, Role: FOLLOWER
I20260619 02:51:03.213645 26412 ts_tablet_manager.cc:1397] T 9a9c3a545dff474284dfa07856c8db6e P 7c852f00331741bf97c3d710d873c775: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.002s
I20260619 02:51:03.214012 26409 consensus_queue.cc:260] T 9a9c3a545dff474284dfa07856c8db6e P f46cfab95e6e44c0be837c69674063c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10455eb59a7040a98cb6dc8988bc24b8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45679 } } peers { permanent_uuid: "f46cfab95e6e44c0be837c69674063c9" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45195 } } peers { permanent_uuid: "7c852f00331741bf97c3d710d873c775" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34939 } }
I20260619 02:51:03.216089 26409 ts_tablet_manager.cc:1428] T 9a9c3a545dff474284dfa07856c8db6e P f46cfab95e6e44c0be837c69674063c9: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260619 02:51:03.216429 26236 heartbeater.cc:503] Master 127.18.134.62:33075 was elected leader, sending a full tablet report...
I20260619 02:51:03.224282 26411 raft_consensus.cc:348] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10455eb59a7040a98cb6dc8988bc24b8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45679 } } peers { permanent_uuid: "f46cfab95e6e44c0be837c69674063c9" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45195 } } peers { permanent_uuid: "7c852f00331741bf97c3d710d873c775" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34939 } }
I20260619 02:51:03.224786 26411 raft_consensus.cc:374] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:03.224609 26412 raft_consensus.cc:348] T 9a9c3a545dff474284dfa07856c8db6e P 7c852f00331741bf97c3d710d873c775 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10455eb59a7040a98cb6dc8988bc24b8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45679 } } peers { permanent_uuid: "f46cfab95e6e44c0be837c69674063c9" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45195 } } peers { permanent_uuid: "7c852f00331741bf97c3d710d873c775" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34939 } }
I20260619 02:51:03.224915 26411 raft_consensus.cc:729] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10455eb59a7040a98cb6dc8988bc24b8, State: Initialized, Role: FOLLOWER
I20260619 02:51:03.225049 26412 raft_consensus.cc:374] T 9a9c3a545dff474284dfa07856c8db6e P 7c852f00331741bf97c3d710d873c775 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:03.225136 26412 raft_consensus.cc:729] T 9a9c3a545dff474284dfa07856c8db6e P 7c852f00331741bf97c3d710d873c775 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c852f00331741bf97c3d710d873c775, State: Initialized, Role: FOLLOWER
I20260619 02:51:03.225600 26411 consensus_queue.cc:260] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10455eb59a7040a98cb6dc8988bc24b8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45679 } } peers { permanent_uuid: "f46cfab95e6e44c0be837c69674063c9" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45195 } } peers { permanent_uuid: "7c852f00331741bf97c3d710d873c775" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34939 } }
I20260619 02:51:03.225689 26412 consensus_queue.cc:260] T 9a9c3a545dff474284dfa07856c8db6e P 7c852f00331741bf97c3d710d873c775 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10455eb59a7040a98cb6dc8988bc24b8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45679 } } peers { permanent_uuid: "f46cfab95e6e44c0be837c69674063c9" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45195 } } peers { permanent_uuid: "7c852f00331741bf97c3d710d873c775" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34939 } }
I20260619 02:51:03.227537 26411 ts_tablet_manager.cc:1428] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8: Time spent starting tablet: real 0.015s user 0.002s sys 0.014s
I20260619 02:51:03.227694 26412 ts_tablet_manager.cc:1428] T 9a9c3a545dff474284dfa07856c8db6e P 7c852f00331741bf97c3d710d873c775: Time spent starting tablet: real 0.014s user 0.011s sys 0.002s
I20260619 02:51:03.227950 26377 heartbeater.cc:503] Master 127.18.134.62:33075 was elected leader, sending a full tablet report...
W20260619 02:51:03.248378 26238 tablet.cc:2369] T 9a9c3a545dff474284dfa07856c8db6e P f46cfab95e6e44c0be837c69674063c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:51:03.301333 26379 tablet.cc:2369] T 9a9c3a545dff474284dfa07856c8db6e P 7c852f00331741bf97c3d710d873c775: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:51:03.325537 26416 raft_consensus.cc:482] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:51:03.325881 26416 raft_consensus.cc:504] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10455eb59a7040a98cb6dc8988bc24b8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45679 } } peers { permanent_uuid: "f46cfab95e6e44c0be837c69674063c9" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45195 } } peers { permanent_uuid: "7c852f00331741bf97c3d710d873c775" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34939 } }
I20260619 02:51:03.327585 26416 leader_election.cc:288] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f46cfab95e6e44c0be837c69674063c9 (127.18.134.2:45195), 7c852f00331741bf97c3d710d873c775 (127.18.134.3:34939)
I20260619 02:51:03.333863 26407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:03.327995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:53731 (local address 127.18.134.3:34939)
0619 02:51:03.328182 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:03.328188 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:03.328731 (+ 543us) server_negotiation.cc:407] Connection header received
0619 02:51:03.328909 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:03.328915 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:03.328976 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:03.329039 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:03.329559 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.330282 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.332185 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.332954 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.333097 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:03.333175 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:03.333488 (+ 313us) server_negotiation.cc:299] Negotiation successful
0619 02:51:03.333610 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 02:51:03.333961 26408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:03.328051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:42675 (local address 127.18.134.2:45195)
0619 02:51:03.328258 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:03.328264 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:03.328780 (+ 516us) server_negotiation.cc:407] Connection header received
0619 02:51:03.328928 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:03.328932 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:03.328987 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:03.329064 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:03.329610 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.330297 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.332192 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.332962 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.333125 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:03.333207 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:03.333509 (+ 302us) server_negotiation.cc:299] Negotiation successful
0619 02:51:03.333629 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 02:51:03.334022 26418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:03.327856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:45195 (local address 127.18.134.1:42675)
0619 02:51:03.328661 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:03.328675 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:03.328803 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:03.329089 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:03.329094 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:03.329115 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:03.329464 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:03.329473 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.330486 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.330489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:03.332036 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:03.332047 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.333219 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.333224 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:03.333360 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:03.333383 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 02:51:03.333463 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 02:51:03.333585 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"mutex_wait_us":57,"thread_start_us":277,"threads_started":1}
I20260619 02:51:03.334067 26419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:03.327834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:34939 (local address 127.18.134.1:53731)
0619 02:51:03.328606 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:03.328624 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:03.328763 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:03.329073 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:03.329078 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:03.329100 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:03.329427 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:03.329439 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.330401 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.330406 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:03.332043 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:03.332054 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.333173 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.333179 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:03.333340 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:03.333370 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 02:51:03.333445 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 02:51:03.333575 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"mutex_wait_us":7,"thread_start_us":554,"threads_started":1}
I20260619 02:51:03.335096 26190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a9c3a545dff474284dfa07856c8db6e" candidate_uuid: "10455eb59a7040a98cb6dc8988bc24b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46cfab95e6e44c0be837c69674063c9" is_pre_election: true
I20260619 02:51:03.335032 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a9c3a545dff474284dfa07856c8db6e" candidate_uuid: "10455eb59a7040a98cb6dc8988bc24b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c852f00331741bf97c3d710d873c775" is_pre_election: true
I20260619 02:51:03.335568 26190 raft_consensus.cc:2454] T 9a9c3a545dff474284dfa07856c8db6e P f46cfab95e6e44c0be837c69674063c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10455eb59a7040a98cb6dc8988bc24b8 in term 0.
I20260619 02:51:03.335883 26331 raft_consensus.cc:2454] T 9a9c3a545dff474284dfa07856c8db6e P 7c852f00331741bf97c3d710d873c775 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10455eb59a7040a98cb6dc8988bc24b8 in term 0.
I20260619 02:51:03.336211 25974 leader_election.cc:302] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10455eb59a7040a98cb6dc8988bc24b8, f46cfab95e6e44c0be837c69674063c9; no voters:
I20260619 02:51:03.336627 26416 raft_consensus.cc:2781] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:51:03.336783 26416 raft_consensus.cc:482] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:51:03.336876 26416 raft_consensus.cc:3037] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:03.338981 26416 raft_consensus.cc:504] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10455eb59a7040a98cb6dc8988bc24b8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45679 } } peers { permanent_uuid: "f46cfab95e6e44c0be837c69674063c9" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45195 } } peers { permanent_uuid: "7c852f00331741bf97c3d710d873c775" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34939 } }
I20260619 02:51:03.339567 26416 leader_election.cc:288] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8 [CANDIDATE]: Term 1 election: Requested vote from peers f46cfab95e6e44c0be837c69674063c9 (127.18.134.2:45195), 7c852f00331741bf97c3d710d873c775 (127.18.134.3:34939)
I20260619 02:51:03.340060 26190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a9c3a545dff474284dfa07856c8db6e" candidate_uuid: "10455eb59a7040a98cb6dc8988bc24b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46cfab95e6e44c0be837c69674063c9"
I20260619 02:51:03.340081 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a9c3a545dff474284dfa07856c8db6e" candidate_uuid: "10455eb59a7040a98cb6dc8988bc24b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c852f00331741bf97c3d710d873c775"
I20260619 02:51:03.340252 26190 raft_consensus.cc:3037] T 9a9c3a545dff474284dfa07856c8db6e P f46cfab95e6e44c0be837c69674063c9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:03.340276 26331 raft_consensus.cc:3037] T 9a9c3a545dff474284dfa07856c8db6e P 7c852f00331741bf97c3d710d873c775 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:03.342411 26331 raft_consensus.cc:2454] T 9a9c3a545dff474284dfa07856c8db6e P 7c852f00331741bf97c3d710d873c775 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10455eb59a7040a98cb6dc8988bc24b8 in term 1.
I20260619 02:51:03.342420 26190 raft_consensus.cc:2454] T 9a9c3a545dff474284dfa07856c8db6e P f46cfab95e6e44c0be837c69674063c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10455eb59a7040a98cb6dc8988bc24b8 in term 1.
I20260619 02:51:03.342793 25974 leader_election.cc:302] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10455eb59a7040a98cb6dc8988bc24b8, f46cfab95e6e44c0be837c69674063c9; no voters:
I20260619 02:51:03.343082 26416 raft_consensus.cc:2781] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:51:03.343494 26416 raft_consensus.cc:686] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8 [term 1 LEADER]: Becoming Leader. State: Replica: 10455eb59a7040a98cb6dc8988bc24b8, State: Running, Role: LEADER
I20260619 02:51:03.343990 26416 consensus_queue.cc:237] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10455eb59a7040a98cb6dc8988bc24b8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45679 } } peers { permanent_uuid: "f46cfab95e6e44c0be837c69674063c9" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45195 } } peers { permanent_uuid: "7c852f00331741bf97c3d710d873c775" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34939 } }
I20260619 02:51:03.348670 25119 catalog_manager.cc:5495] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 10455eb59a7040a98cb6dc8988bc24b8 (127.18.134.1). New cstate: current_term: 1 leader_uuid: "10455eb59a7040a98cb6dc8988bc24b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10455eb59a7040a98cb6dc8988bc24b8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f46cfab95e6e44c0be837c69674063c9" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c852f00331741bf97c3d710d873c775" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34939 } health_report { overall_health: UNKNOWN } } }
I20260619 02:51:03.410984 26404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:03.405555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49664 (local address 127.18.134.1:45679)
0619 02:51:03.405744 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:03.405751 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:03.405772 (+ 21us) server_negotiation.cc:407] Connection header received
0619 02:51:03.405895 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:03.405899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:03.405972 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:03.406130 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:51:03.406534 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.407394 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.408766 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.409191 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.409285 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:03.409842 (+ 557us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:51:03.410377 (+ 535us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:51:03.410523 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:03.410718 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 02:51:03.410788 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260619 02:51:03.414880 26088 tablet.cc:2369] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:51:03.419669 26190 raft_consensus.cc:1261] T 9a9c3a545dff474284dfa07856c8db6e P f46cfab95e6e44c0be837c69674063c9 [term 1 FOLLOWER]: Refusing update from remote peer 10455eb59a7040a98cb6dc8988bc24b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:51:03.419817 26331 raft_consensus.cc:1261] T 9a9c3a545dff474284dfa07856c8db6e P 7c852f00331741bf97c3d710d873c775 [term 1 FOLLOWER]: Refusing update from remote peer 10455eb59a7040a98cb6dc8988bc24b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:51:03.420727 26416 consensus_queue.cc:1035] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f46cfab95e6e44c0be837c69674063c9" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:51:03.421018 26420 consensus_queue.cc:1035] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c852f00331741bf97c3d710d873c775" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:51:03.432156 26424 mvcc.cc:205] Tried to move back new op lower bound from 7298406250156769280 to 7298406249860374528. Current Snapshot: MvccSnapshot[applied={T|T < 7298406250156769280}]
I20260619 02:51:03.453816 25119 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:51:03.454092 25119 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:51:03.462926 25119 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0094edcfb4284971a6d45b1ec4e48a30: Sending DeleteTablet for 3 replicas of tablet 9a9c3a545dff474284dfa07856c8db6e
I20260619 02:51:03.464071 26170 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a9c3a545dff474284dfa07856c8db6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:51:03 UTC) from {username='oryx', principal='oryx/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:38186
I20260619 02:51:03.464993 26433 tablet_replica.cc:330] T 9a9c3a545dff474284dfa07856c8db6e P f46cfab95e6e44c0be837c69674063c9: stopping tablet replica
I20260619 02:51:03.465561 26433 raft_consensus.cc:2229] T 9a9c3a545dff474284dfa07856c8db6e P f46cfab95e6e44c0be837c69674063c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:51:03.466713 26433 raft_consensus.cc:2258] T 9a9c3a545dff474284dfa07856c8db6e P f46cfab95e6e44c0be837c69674063c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:51:03.470264 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 25088
I20260619 02:51:03.471266 25150 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:03.472807 26433 ts_tablet_manager.cc:1905] T 9a9c3a545dff474284dfa07856c8db6e P f46cfab95e6e44c0be837c69674063c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:51:03.480296 26433 ts_tablet_manager.cc:1918] T 9a9c3a545dff474284dfa07856c8db6e P f46cfab95e6e44c0be837c69674063c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:51:03.480455 26433 log.cc:1196] T 9a9c3a545dff474284dfa07856c8db6e P f46cfab95e6e44c0be837c69674063c9: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/wal/wals/9a9c3a545dff474284dfa07856c8db6e
I20260619 02:51:03.480983 26433 ts_tablet_manager.cc:1939] T 9a9c3a545dff474284dfa07856c8db6e P f46cfab95e6e44c0be837c69674063c9: Deleting consensus metadata
I20260619 02:51:03.491405 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:03.468562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50528 (local address 127.18.134.61:35581)
0619 02:51:03.468762 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:03.468769 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:03.468788 (+ 19us) server_negotiation.cc:407] Connection header received
0619 02:51:03.468845 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:03.468851 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:03.468928 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:03.469027 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:03.471889 (+ 2862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.473052 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.475060 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.476298 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.476402 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:03.485007 (+ 8605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:03.485015 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:03.485020 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:03.485063 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:03.488654 (+ 3591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:03.489469 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:03.489475 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:03.489479 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:03.489611 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:03.490010 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:03.490014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:03.490016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:03.490299 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:03.490511 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:03.490880 (+ 369us) server_negotiation.cc:299] Negotiation successful
0619 02:51:03.491078 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 02:51:03.507527 26394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:03.469505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47660 (local address 127.18.134.60:44329)
0619 02:51:03.469765 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:03.469771 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:03.491586 (+ 21815us) server_negotiation.cc:407] Connection header received
0619 02:51:03.491827 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:03.491833 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:03.491927 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:03.491994 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:03.493919 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.495011 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.496995 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.497601 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.497705 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:03.501495 (+ 3790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:03.501501 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:03.501507 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:03.501544 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:03.504687 (+ 3143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:03.505388 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:03.505394 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:03.505398 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:03.505527 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:03.505969 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:03.505975 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:03.505978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:03.506272 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:03.506465 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:03.506998 (+ 533us) server_negotiation.cc:299] Negotiation successful
0619 02:51:03.507211 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260619 02:51:03.671977 25105 catalog_manager.cc:4841] TS f46cfab95e6e44c0be837c69674063c9 (127.18.134.2:45195): tablet 9a9c3a545dff474284dfa07856c8db6e (table test-table [id=b0b5ad99a9f746279e54917d22795e1d]) successfully deleted
W20260619 02:51:03.672241 25105 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.18.134.2:45195, user_credentials={real_user=slave}} blocked reactor thread for 189406us
I20260619 02:51:03.672991 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 25088
I20260619 02:51:03.710563 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:03.694148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50532 (local address 127.18.134.61:35581)
0619 02:51:03.694324 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:03.694331 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:03.694978 (+ 647us) server_negotiation.cc:407] Connection header received
0619 02:51:03.695242 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:03.695248 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:03.695328 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:03.695395 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:03.697526 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.698401 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.700103 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.700749 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.700879 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:03.703993 (+ 3114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:03.703999 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:03.704003 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:03.704038 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:03.707444 (+ 3406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:03.708329 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:03.708335 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:03.708339 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:03.708495 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:03.708933 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:03.708938 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:03.708943 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:03.709345 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:03.709655 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:03.710003 (+ 348us) server_negotiation.cc:299] Negotiation successful
0619 02:51:03.710231 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 02:51:03.711251 26394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:03.695536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47674 (local address 127.18.134.60:44329)
0619 02:51:03.695731 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:03.695738 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:03.695759 (+ 21us) server_negotiation.cc:407] Connection header received
0619 02:51:03.695824 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:03.695830 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:03.695908 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:03.696008 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:03.697918 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.699028 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.700663 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:03.701268 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:03.701412 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:03.704529 (+ 3117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:03.704536 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:03.704540 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:03.704574 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:03.707749 (+ 3175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:03.708563 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:03.708569 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:03.708573 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:03.708722 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:03.709183 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:03.709187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:03.709189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:03.709568 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:03.709847 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:03.710697 (+ 850us) server_negotiation.cc:299] Negotiation successful
0619 02:51:03.710940 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260619 02:51:03.905882 25974 consensus_peers.cc:487] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8 -> Peer f46cfab95e6e44c0be837c69674063c9 (127.18.134.2:45195): Couldn't send request to peer f46cfab95e6e44c0be837c69674063c9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a9c3a545dff474284dfa07856c8db6e. This is attempt 1: this message will repeat every 5th retry.
I20260619 02:51:04.367816 26446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:04.366933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.18.134.1:51779)
0619 02:51:04.367470 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:04.367602 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":287,"threads_started":1}
W20260619 02:51:04.368278 26086 heartbeater.cc:644] Failed to heartbeat to 127.18.134.62:33075 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
I20260619 02:51:04.439978 26447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:04.439170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.18.134.3:57901)
0619 02:51:04.439611 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:04.439754 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":217,"threads_started":1}
W20260619 02:51:04.440379 26377 heartbeater.cc:644] Failed to heartbeat to 127.18.134.62:33075 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
I20260619 02:51:04.470319 26448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:04.469646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.18.134.2:35475)
0619 02:51:04.470032 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:04.470143 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":184,"threads_started":1}
W20260619 02:51:04.470736 26236 heartbeater.cc:644] Failed to heartbeat to 127.18.134.62:33075 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
I20260619 02:51:04.839902 26446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:04.839458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.18.134.1:58065)
0619 02:51:04.839615 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:04.839729 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 02:51:04.959708 26449 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0094edcfb4284971a6d45b1ec4e48a30)
I20260619 02:51:04.960042 26449 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: false } }
I20260619 02:51:04.961351 26449 leader_election.cc:288] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0094edcfb4284971a6d45b1ec4e48a30 (127.18.134.62:33075), 59aa8b8f204d41a9b88df69b6eca76d3 (127.18.134.60:44329)
I20260619 02:51:04.962944 26450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:04.961562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.0.0.1:38248)
0619 02:51:04.962033 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:04.962188 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":218,"threads_started":1}
I20260619 02:51:04.962996 26453 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0094edcfb4284971a6d45b1ec4e48a30)
W20260619 02:51:04.963200 25702 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
I20260619 02:51:04.963318 26453 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: false } }
I20260619 02:51:04.964941 26453 leader_election.cc:288] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0094edcfb4284971a6d45b1ec4e48a30 (127.18.134.62:33075), 9a94a362219f40ec98932b3e1f58a9cd (127.18.134.61:35581)
I20260619 02:51:04.967319 26450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:04.966827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.0.0.1:38270)
0619 02:51:04.967047 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:04.967160 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260619 02:51:04.967576 25702 leader_election.cc:334] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0094edcfb4284971a6d45b1ec4e48a30 (127.18.134.62:33075): Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
I20260619 02:51:04.968418 26455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:04.965427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.0.0.1:38264)
0619 02:51:04.967903 (+ 2476us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:04.968123 (+ 220us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2367,"thread_start_us":999,"threads_started":1}
W20260619 02:51:04.968750 25889 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
I20260619 02:51:04.970374 26455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:04.969934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.0.0.1:38280)
0619 02:51:04.970137 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:04.970212 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
W20260619 02:51:04.970582 25889 leader_election.cc:334] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0094edcfb4284971a6d45b1ec4e48a30 (127.18.134.62:33075): Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
I20260619 02:51:04.972769 26452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:04.962430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47680 (local address 127.18.134.60:44329)
0619 02:51:04.965451 (+ 3021us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:04.965459 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:04.965482 (+ 23us) server_negotiation.cc:407] Connection header received
0619 02:51:04.965562 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:04.965568 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:04.965648 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:04.965758 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:04.967277 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:04.968314 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:04.970494 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:04.971866 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:04.971975 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:04.972068 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:04.972335 (+ 267us) server_negotiation.cc:299] Negotiation successful
0619 02:51:04.972462 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":2901,"thread_start_us":2737,"threads_started":1}
I20260619 02:51:04.972784 26451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:04.962188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44329 (local address 127.0.0.1:47680)
0619 02:51:04.962702 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:04.962722 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:04.962868 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:04.966704 (+ 3836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:04.966710 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:04.966744 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:04.967054 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:04.967066 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:04.968497 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:04.968502 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:04.970353 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:04.970369 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:04.972015 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:04.972021 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:04.972196 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:04.972220 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 02:51:04.972309 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 02:51:04.972450 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":251,"threads_started":1}
I20260619 02:51:04.973371 25914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9a94a362219f40ec98932b3e1f58a9cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" is_pre_election: true
I20260619 02:51:04.973690 25914 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a94a362219f40ec98932b3e1f58a9cd in term 1.
I20260619 02:51:04.973778 26456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:04.965250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:35581 (local address 127.0.0.1:50534)
0619 02:51:04.967023 (+ 1773us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:04.967041 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:04.967145 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:04.968263 (+ 1118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:04.968269 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:04.968300 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:04.968642 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:04.968651 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:04.970013 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:04.970018 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:04.971927 (+ 1909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:04.971939 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:04.973123 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:04.973131 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:04.973288 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:04.973310 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 02:51:04.973383 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 02:51:04.973468 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1289,"mutex_wait_us":54,"thread_start_us":2401,"threads_started":1}
I20260619 02:51:04.974169 25702 leader_election.cc:302] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 59aa8b8f204d41a9b88df69b6eca76d3, 9a94a362219f40ec98932b3e1f58a9cd; no voters: 0094edcfb4284971a6d45b1ec4e48a30
I20260619 02:51:04.974334 26457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:04.965539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50534 (local address 127.18.134.61:35581)
0619 02:51:04.967978 (+ 2439us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:04.967983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:04.968004 (+ 21us) server_negotiation.cc:407] Connection header received
0619 02:51:04.968059 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:04.968064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:04.968127 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:04.968225 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:04.968808 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:04.969889 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:04.972061 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:04.972929 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:04.973742 (+ 813us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:04.973836 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:04.973883 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 02:51:04.973999 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2338,"thread_start_us":1813,"threads_started":1}
I20260619 02:51:04.974525 26449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 02:51:04.974678 26449 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0094edcfb4284971a6d45b1ec4e48a30)
I20260619 02:51:04.974781 26449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:51:04.976862 26449 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: false } }
I20260619 02:51:04.977471 26449 leader_election.cc:288] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [CANDIDATE]: Term 2 election: Requested vote from peers 0094edcfb4284971a6d45b1ec4e48a30 (127.18.134.62:33075), 59aa8b8f204d41a9b88df69b6eca76d3 (127.18.134.60:44329)
I20260619 02:51:04.977600 25727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9a94a362219f40ec98932b3e1f58a9cd" is_pre_election: true
I20260619 02:51:04.978003 26451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:04.977585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.0.0.1:38288)
0619 02:51:04.977742 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:04.977853 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 02:51:04.978050 25727 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 59aa8b8f204d41a9b88df69b6eca76d3 in current term 2: Already voted for candidate 9a94a362219f40ec98932b3e1f58a9cd in this term.
I20260619 02:51:04.978287 25914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9a94a362219f40ec98932b3e1f58a9cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "59aa8b8f204d41a9b88df69b6eca76d3"
I20260619 02:51:04.978528 25914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:51:04.978699 25890 leader_election.cc:302] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 59aa8b8f204d41a9b88df69b6eca76d3; no voters: 0094edcfb4284971a6d45b1ec4e48a30, 9a94a362219f40ec98932b3e1f58a9cd
I20260619 02:51:04.979056 26451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:04.978689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.0.0.1:38300)
0619 02:51:04.978837 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:04.978924 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260619 02:51:04.979197 25702 leader_election.cc:334] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0094edcfb4284971a6d45b1ec4e48a30 (127.18.134.62:33075): Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
I20260619 02:51:04.980826 25914 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a94a362219f40ec98932b3e1f58a9cd in term 2.
I20260619 02:51:04.981014 26453 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 02:51:04.981207 25702 leader_election.cc:302] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 59aa8b8f204d41a9b88df69b6eca76d3, 9a94a362219f40ec98932b3e1f58a9cd; no voters: 0094edcfb4284971a6d45b1ec4e48a30
I20260619 02:51:04.981452 26449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [term 2 FOLLOWER]: Leader election won for term 2
I20260619 02:51:04.981925 26449 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [term 2 LEADER]: Becoming Leader. State: Replica: 9a94a362219f40ec98932b3e1f58a9cd, State: Running, Role: LEADER
I20260619 02:51:04.982276 26449 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: false } }
I20260619 02:51:04.984025 26459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a94a362219f40ec98932b3e1f58a9cd. Latest consensus state: current_term: 2 leader_uuid: "9a94a362219f40ec98932b3e1f58a9cd" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: false } } }
I20260619 02:51:04.984344 26459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [sys.catalog]: This master's current role is: LEADER
I20260619 02:51:05.407038 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.405420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.18.134.1:35887)
0619 02:51:05.406772 (+ 1352us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:05.406856 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1240,"thread_start_us":1143,"threads_started":1}
I20260619 02:51:05.441787 26466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.441058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.18.134.3:43187)
0619 02:51:05.441539 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:05.441623 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":237,"threads_started":1}
I20260619 02:51:05.472165 26467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.471490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.18.134.2:36305)
0619 02:51:05.471907 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:05.471993 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":200,"threads_started":1}
I20260619 02:51:05.552493 25914 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [term 2 FOLLOWER]: Refusing update from remote peer 9a94a362219f40ec98932b3e1f58a9cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 13. (index mismatch)
I20260619 02:51:05.553174 26468 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260619 02:51:05.555078 26469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a94a362219f40ec98932b3e1f58a9cd. Latest consensus state: current_term: 2 leader_uuid: "9a94a362219f40ec98932b3e1f58a9cd" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: false } } }
I20260619 02:51:05.555379 26469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:51:05.557420 26468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "9a94a362219f40ec98932b3e1f58a9cd" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: false } } }
I20260619 02:51:05.557650 26468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd [sys.catalog]: This master's current role is: LEADER
I20260619 02:51:05.557839 26469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "9a94a362219f40ec98932b3e1f58a9cd" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0094edcfb4284971a6d45b1ec4e48a30" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33075 } } peers { permanent_uuid: "9a94a362219f40ec98932b3e1f58a9cd" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35581 } attrs { promote: false } } peers { permanent_uuid: "59aa8b8f204d41a9b88df69b6eca76d3" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44329 } attrs { promote: false } } }
I20260619 02:51:05.558063 26469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59aa8b8f204d41a9b88df69b6eca76d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:51:05.558349 26475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.557455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.0.0.1:38428)
0619 02:51:05.557984 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:05.558112 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":505,"threads_started":1}
I20260619 02:51:05.559633 26475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.559303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.0.0.1:38430)
0619 02:51:05.559469 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:05.559532 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260619 02:51:05.559800 25702 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd -> Peer 0094edcfb4284971a6d45b1ec4e48a30 (127.18.134.62:33075): Couldn't send request to peer 0094edcfb4284971a6d45b1ec4e48a30. Status: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 02:51:05.629631 26461 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:51:05.633261 26461 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:51:05.634047 26461 catalog_manager.cc:1259] Loaded cluster ID: 9cbe8949e88b4785a8e697bf8dceef26
I20260619 02:51:05.634122 26461 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:51:05.635224 26461 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:51:05.635957 26461 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd: Loaded TSK: 0
I20260619 02:51:05.636747 26461 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:51:05.675199 25717 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50532:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:51:05.681905 25717 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:51:05.710276 26475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.695563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:45195 (local address 127.0.0.1:38192)
0619 02:51:05.695701 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:05.695720 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:05.695823 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:05.703750 (+ 7927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:05.703755 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:05.703780 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:05.704007 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:05.704019 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:05.705556 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:05.705562 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:05.707697 (+ 2135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:05.707713 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:05.709447 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:05.709455 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:05.709667 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:05.709705 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 02:51:05.709771 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 02:51:05.709928 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 02:51:05.712523 26479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.695685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38192 (local address 127.18.134.2:45195)
0619 02:51:05.703488 (+ 7803us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:05.703492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:05.703513 (+ 21us) server_negotiation.cc:407] Connection header received
0619 02:51:05.703567 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:05.703570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:05.703622 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:05.703716 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:05.704183 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:05.705380 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:05.707927 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:05.709246 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:05.711834 (+ 2588us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:05.711958 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:05.712014 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 02:51:05.712165 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7690,"thread_start_us":7588,"threads_started":1}
I20260619 02:51:05.713451 26170 tablet_service.cc:1467] Processing CreateTablet for tablet 7a757b0c68de4143b74065852393fa13 (DEFAULT_TABLE table=test-table [id=adec3170d05d42ac8767aae4b47cc5d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:51:05.713954 26170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a757b0c68de4143b74065852393fa13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:05.719058 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.697154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55616 (local address 127.18.134.3:34939)
0619 02:51:05.712829 (+ 15675us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:05.712835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:05.712853 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:51:05.712917 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:05.712922 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:05.713001 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:05.713100 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:05.713515 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:05.714452 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:05.716813 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:05.718068 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:05.718196 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:05.718309 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:05.718566 (+ 257us) server_negotiation.cc:299] Negotiation successful
0619 02:51:05.718753 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15561,"thread_start_us":15440,"threads_started":1}
I20260619 02:51:05.719107 26475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.697028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:34939 (local address 127.0.0.1:55616)
0619 02:51:05.710529 (+ 13501us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:05.710550 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:05.710670 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:05.713141 (+ 2471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:05.713146 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:05.713171 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:05.713335 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:05.713344 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:05.714650 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:05.714656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:05.716657 (+ 2001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:05.716670 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:05.718232 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:05.718238 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:05.718406 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:05.718438 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 02:51:05.718537 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 02:51:05.718717 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13406,"mutex_wait_us":21,"thread_start_us":19467,"threads_started":1}
I20260619 02:51:05.719280 26480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.695891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:45679 (local address 127.0.0.1:49680)
0619 02:51:05.696380 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:05.696401 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:05.696511 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:05.712006 (+ 15495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:05.712010 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:05.712034 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:05.712273 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:05.712282 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:05.713688 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:05.713694 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:05.716890 (+ 3196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:05.716901 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:05.718503 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:05.718508 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:05.718708 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:05.718736 (+ 28us) client_negotiation.cc:769] Sending connection context
0619 02:51:05.718813 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 02:51:05.718937 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":26,"thread_start_us":260,"threads_started":1}
I20260619 02:51:05.719826 26481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.696283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49680 (local address 127.18.134.1:45679)
0619 02:51:05.711710 (+ 15427us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:05.711716 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:05.711735 (+ 19us) server_negotiation.cc:407] Connection header received
0619 02:51:05.711802 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:05.711807 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:05.711887 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:05.711990 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:05.712446 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:05.713500 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:05.717051 (+ 3551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:05.718169 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:05.718302 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:05.718397 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:05.719465 (+ 1068us) server_negotiation.cc:299] Negotiation successful
0619 02:51:05.719585 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15279,"thread_start_us":15817,"threads_started":1}
I20260619 02:51:05.720470 26020 tablet_service.cc:1467] Processing CreateTablet for tablet 7a757b0c68de4143b74065852393fa13 (DEFAULT_TABLE table=test-table [id=adec3170d05d42ac8767aae4b47cc5d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:51:05.720692 26484 tablet_bootstrap.cc:492] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9: Bootstrap starting.
I20260619 02:51:05.720858 26020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a757b0c68de4143b74065852393fa13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:05.720885 26311 tablet_service.cc:1467] Processing CreateTablet for tablet 7a757b0c68de4143b74065852393fa13 (DEFAULT_TABLE table=test-table [id=adec3170d05d42ac8767aae4b47cc5d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:51:05.721237 26311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a757b0c68de4143b74065852393fa13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:05.722185 26484 tablet_bootstrap.cc:654] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:05.724009 26484 tablet_bootstrap.cc:492] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9: No bootstrap required, opened a new log
I20260619 02:51:05.724212 26484 ts_tablet_manager.cc:1397] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260619 02:51:05.725044 26484 raft_consensus.cc:348] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46cfab95e6e44c0be837c69674063c9" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45195 } } peers { permanent_uuid: "10455eb59a7040a98cb6dc8988bc24b8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45679 } } peers { permanent_uuid: "7c852f00331741bf97c3d710d873c775" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34939 } }
I20260619 02:51:05.725276 26484 raft_consensus.cc:374] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:05.725380 26484 raft_consensus.cc:729] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46cfab95e6e44c0be837c69674063c9, State: Initialized, Role: FOLLOWER
I20260619 02:51:05.725610 26484 consensus_queue.cc:260] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46cfab95e6e44c0be837c69674063c9" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45195 } } peers { permanent_uuid: "10455eb59a7040a98cb6dc8988bc24b8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45679 } } peers { permanent_uuid: "7c852f00331741bf97c3d710d873c775" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34939 } }
I20260619 02:51:05.726393 26484 ts_tablet_manager.cc:1428] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 02:51:05.727717 26467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.727045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.18.134.2:43665)
0619 02:51:05.727484 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:05.727577 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329}
I20260619 02:51:05.727843 26488 tablet_bootstrap.cc:492] T 7a757b0c68de4143b74065852393fa13 P 7c852f00331741bf97c3d710d873c775: Bootstrap starting.
I20260619 02:51:05.729508 26488 tablet_bootstrap.cc:654] T 7a757b0c68de4143b74065852393fa13 P 7c852f00331741bf97c3d710d873c775: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:05.731290 26237 heartbeater.cc:503] Master 127.18.134.61:35581 was elected leader, sending a full tablet report...
I20260619 02:51:05.731967 26467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.731562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.18.134.2:37969)
0619 02:51:05.731708 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:05.731803 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 02:51:05.733551 26488 tablet_bootstrap.cc:492] T 7a757b0c68de4143b74065852393fa13 P 7c852f00331741bf97c3d710d873c775: No bootstrap required, opened a new log
I20260619 02:51:05.733752 26488 ts_tablet_manager.cc:1397] T 7a757b0c68de4143b74065852393fa13 P 7c852f00331741bf97c3d710d873c775: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 02:51:05.734556 26488 raft_consensus.cc:348] T 7a757b0c68de4143b74065852393fa13 P 7c852f00331741bf97c3d710d873c775 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46cfab95e6e44c0be837c69674063c9" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45195 } } peers { permanent_uuid: "10455eb59a7040a98cb6dc8988bc24b8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45679 } } peers { permanent_uuid: "7c852f00331741bf97c3d710d873c775" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34939 } }
I20260619 02:51:05.734784 26488 raft_consensus.cc:374] T 7a757b0c68de4143b74065852393fa13 P 7c852f00331741bf97c3d710d873c775 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:05.734876 26488 raft_consensus.cc:729] T 7a757b0c68de4143b74065852393fa13 P 7c852f00331741bf97c3d710d873c775 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c852f00331741bf97c3d710d873c775, State: Initialized, Role: FOLLOWER
I20260619 02:51:05.735175 26487 tablet_bootstrap.cc:492] T 7a757b0c68de4143b74065852393fa13 P 10455eb59a7040a98cb6dc8988bc24b8: Bootstrap starting.
I20260619 02:51:05.735090 26488 consensus_queue.cc:260] T 7a757b0c68de4143b74065852393fa13 P 7c852f00331741bf97c3d710d873c775 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46cfab95e6e44c0be837c69674063c9" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45195 } } peers { permanent_uuid: "10455eb59a7040a98cb6dc8988bc24b8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45679 } } peers { permanent_uuid: "7c852f00331741bf97c3d710d873c775" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34939 } }
I20260619 02:51:05.735868 26488 ts_tablet_manager.cc:1428] T 7a757b0c68de4143b74065852393fa13 P 7c852f00331741bf97c3d710d873c775: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:51:05.736682 26466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.736177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.18.134.3:52927)
0619 02:51:05.736442 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:05.736533 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
I20260619 02:51:05.736889 26487 tablet_bootstrap.cc:654] T 7a757b0c68de4143b74065852393fa13 P 10455eb59a7040a98cb6dc8988bc24b8: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:05.738238 26378 heartbeater.cc:503] Master 127.18.134.61:35581 was elected leader, sending a full tablet report...
I20260619 02:51:05.738567 26466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.738176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.18.134.3:58939)
0619 02:51:05.738333 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:05.738418 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260619 02:51:05.739235 26487 tablet_bootstrap.cc:492] T 7a757b0c68de4143b74065852393fa13 P 10455eb59a7040a98cb6dc8988bc24b8: No bootstrap required, opened a new log
I20260619 02:51:05.739408 26487 ts_tablet_manager.cc:1397] T 7a757b0c68de4143b74065852393fa13 P 10455eb59a7040a98cb6dc8988bc24b8: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260619 02:51:05.740402 26487 raft_consensus.cc:348] T 7a757b0c68de4143b74065852393fa13 P 10455eb59a7040a98cb6dc8988bc24b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46cfab95e6e44c0be837c69674063c9" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45195 } } peers { permanent_uuid: "10455eb59a7040a98cb6dc8988bc24b8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45679 } } peers { permanent_uuid: "7c852f00331741bf97c3d710d873c775" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34939 } }
I20260619 02:51:05.740658 26487 raft_consensus.cc:374] T 7a757b0c68de4143b74065852393fa13 P 10455eb59a7040a98cb6dc8988bc24b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:05.740796 26487 raft_consensus.cc:729] T 7a757b0c68de4143b74065852393fa13 P 10455eb59a7040a98cb6dc8988bc24b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10455eb59a7040a98cb6dc8988bc24b8, State: Initialized, Role: FOLLOWER
I20260619 02:51:05.741027 26311 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a9c3a545dff474284dfa07856c8db6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:51:03 UTC) from {username='oryx', principal='oryx/127.18.134.61@KRBTEST.COM'} at 127.0.0.1:55616
I20260619 02:51:05.741027 26487 consensus_queue.cc:260] T 7a757b0c68de4143b74065852393fa13 P 10455eb59a7040a98cb6dc8988bc24b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46cfab95e6e44c0be837c69674063c9" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45195 } } peers { permanent_uuid: "10455eb59a7040a98cb6dc8988bc24b8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45679 } } peers { permanent_uuid: "7c852f00331741bf97c3d710d873c775" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34939 } }
I20260619 02:51:05.741564 26487 ts_tablet_manager.cc:1428] T 7a757b0c68de4143b74065852393fa13 P 10455eb59a7040a98cb6dc8988bc24b8: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:51:05.742296 26492 tablet_replica.cc:330] T 9a9c3a545dff474284dfa07856c8db6e P 7c852f00331741bf97c3d710d873c775: stopping tablet replica
I20260619 02:51:05.742394 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.741910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.18.134.1:35637)
0619 02:51:05.742168 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:05.742252 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
I20260619 02:51:05.742870 26492 raft_consensus.cc:2229] T 9a9c3a545dff474284dfa07856c8db6e P 7c852f00331741bf97c3d710d873c775 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:51:05.743333 26492 raft_consensus.cc:2258] T 9a9c3a545dff474284dfa07856c8db6e P 7c852f00331741bf97c3d710d873c775 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:51:05.743450 26087 heartbeater.cc:503] Master 127.18.134.61:35581 was elected leader, sending a full tablet report...
I20260619 02:51:05.744673 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.744141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.18.134.1:60215)
0619 02:51:05.744426 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:05.744523 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194}
I20260619 02:51:05.744956 26492 ts_tablet_manager.cc:1905] T 9a9c3a545dff474284dfa07856c8db6e P 7c852f00331741bf97c3d710d873c775: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:51:05.745955 26020 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a9c3a545dff474284dfa07856c8db6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:51:03 UTC) from {username='oryx', principal='oryx/127.18.134.61@KRBTEST.COM'} at 127.0.0.1:49680
I20260619 02:51:05.747000 26493 tablet_replica.cc:330] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8: stopping tablet replica
I20260619 02:51:05.747344 26493 raft_consensus.cc:2229] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:51:05.747766 26493 raft_consensus.cc:2258] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:51:05.748917 26493 ts_tablet_manager.cc:1905] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:51:05.750109 26492 ts_tablet_manager.cc:1918] T 9a9c3a545dff474284dfa07856c8db6e P 7c852f00331741bf97c3d710d873c775: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:51:05.750257 26492 log.cc:1196] T 9a9c3a545dff474284dfa07856c8db6e P 7c852f00331741bf97c3d710d873c775: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/wal/wals/9a9c3a545dff474284dfa07856c8db6e
I20260619 02:51:05.750663 26492 ts_tablet_manager.cc:1939] T 9a9c3a545dff474284dfa07856c8db6e P 7c852f00331741bf97c3d710d873c775: Deleting consensus metadata
I20260619 02:51:05.752159 25703 catalog_manager.cc:4841] TS 7c852f00331741bf97c3d710d873c775 (127.18.134.3:34939): tablet 9a9c3a545dff474284dfa07856c8db6e (table test-table [id=b0b5ad99a9f746279e54917d22795e1d]) successfully deleted
I20260619 02:51:05.753742 26493 ts_tablet_manager.cc:1918] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:51:05.753890 26493 log.cc:1196] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/wal/wals/9a9c3a545dff474284dfa07856c8db6e
I20260619 02:51:05.754303 26493 ts_tablet_manager.cc:1939] T 9a9c3a545dff474284dfa07856c8db6e P 10455eb59a7040a98cb6dc8988bc24b8: Deleting consensus metadata
I20260619 02:51:05.755610 25703 catalog_manager.cc:4841] TS 10455eb59a7040a98cb6dc8988bc24b8 (127.18.134.1:45679): tablet 9a9c3a545dff474284dfa07856c8db6e (table test-table [id=b0b5ad99a9f746279e54917d22795e1d]) successfully deleted
I20260619 02:51:05.866693 26486 raft_consensus.cc:482] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:51:05.866925 26486 raft_consensus.cc:504] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46cfab95e6e44c0be837c69674063c9" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45195 } } peers { permanent_uuid: "10455eb59a7040a98cb6dc8988bc24b8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45679 } } peers { permanent_uuid: "7c852f00331741bf97c3d710d873c775" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34939 } }
I20260619 02:51:05.868372 26486 leader_election.cc:288] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10455eb59a7040a98cb6dc8988bc24b8 (127.18.134.1:45679), 7c852f00331741bf97c3d710d873c775 (127.18.134.3:34939)
I20260619 02:51:05.874583 26467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.868652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:45679 (local address 127.18.134.2:36845)
0619 02:51:05.868848 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:05.868867 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:05.868995 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:05.869500 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:05.869505 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:05.869536 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:05.869793 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:05.869805 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:05.871035 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:05.871041 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:05.872572 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:05.872584 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:05.873919 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:05.873925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:05.874091 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:05.874126 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 02:51:05.874222 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 02:51:05.874347 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"mutex_wait_us":2}
I20260619 02:51:05.874702 26481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.868827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:36845 (local address 127.18.134.1:45679)
0619 02:51:05.869034 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:05.869040 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:05.869063 (+ 23us) server_negotiation.cc:407] Connection header received
0619 02:51:05.869133 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:05.869138 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:05.869212 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:05.869315 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:05.869942 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:05.870915 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:05.872744 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:05.873709 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:05.873837 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:05.873901 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:05.874254 (+ 353us) server_negotiation.cc:299] Negotiation successful
0619 02:51:05.874382 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 02:51:05.874907 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.869192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:36309 (local address 127.18.134.3:34939)
0619 02:51:05.869462 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:05.869468 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:05.869716 (+ 248us) server_negotiation.cc:407] Connection header received
0619 02:51:05.869829 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:05.869834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:05.869905 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:05.869977 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:05.870528 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:05.871362 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:05.873139 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:05.874011 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:05.874155 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:05.874233 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:05.874561 (+ 328us) server_negotiation.cc:299] Negotiation successful
0619 02:51:05.874703 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260619 02:51:05.875020 26494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.869048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:34939 (local address 127.18.134.2:36309)
0619 02:51:05.869622 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:05.869638 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:05.869733 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:05.870012 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:05.870016 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:05.870040 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:05.870394 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:05.870405 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:05.871508 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:05.871512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:05.872994 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:05.873007 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:05.874216 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:05.874223 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:05.874403 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:05.874434 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 02:51:05.874525 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 02:51:05.874667 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"mutex_wait_us":3,"thread_start_us":305,"threads_started":1}
I20260619 02:51:05.875584 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a757b0c68de4143b74065852393fa13" candidate_uuid: "f46cfab95e6e44c0be837c69674063c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c852f00331741bf97c3d710d873c775" is_pre_election: true
I20260619 02:51:05.875810 26331 raft_consensus.cc:2454] T 7a757b0c68de4143b74065852393fa13 P 7c852f00331741bf97c3d710d873c775 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f46cfab95e6e44c0be837c69674063c9 in term 0.
I20260619 02:51:05.875742 26040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a757b0c68de4143b74065852393fa13" candidate_uuid: "f46cfab95e6e44c0be837c69674063c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10455eb59a7040a98cb6dc8988bc24b8" is_pre_election: true
I20260619 02:51:05.876098 26040 raft_consensus.cc:2454] T 7a757b0c68de4143b74065852393fa13 P 10455eb59a7040a98cb6dc8988bc24b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f46cfab95e6e44c0be837c69674063c9 in term 0.
I20260619 02:51:05.876255 26125 leader_election.cc:302] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c852f00331741bf97c3d710d873c775, f46cfab95e6e44c0be837c69674063c9; no voters:
I20260619 02:51:05.876632 26486 raft_consensus.cc:2781] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:51:05.876755 26486 raft_consensus.cc:482] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:51:05.876853 26486 raft_consensus.cc:3037] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:05.878856 26486 raft_consensus.cc:504] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46cfab95e6e44c0be837c69674063c9" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45195 } } peers { permanent_uuid: "10455eb59a7040a98cb6dc8988bc24b8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45679 } } peers { permanent_uuid: "7c852f00331741bf97c3d710d873c775" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34939 } }
I20260619 02:51:05.879383 26486 leader_election.cc:288] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9 [CANDIDATE]: Term 1 election: Requested vote from peers 10455eb59a7040a98cb6dc8988bc24b8 (127.18.134.1:45679), 7c852f00331741bf97c3d710d873c775 (127.18.134.3:34939)
I20260619 02:51:05.879793 26040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a757b0c68de4143b74065852393fa13" candidate_uuid: "f46cfab95e6e44c0be837c69674063c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10455eb59a7040a98cb6dc8988bc24b8"
I20260619 02:51:05.879858 26331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a757b0c68de4143b74065852393fa13" candidate_uuid: "f46cfab95e6e44c0be837c69674063c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c852f00331741bf97c3d710d873c775"
I20260619 02:51:05.879958 26040 raft_consensus.cc:3037] T 7a757b0c68de4143b74065852393fa13 P 10455eb59a7040a98cb6dc8988bc24b8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:05.880029 26331 raft_consensus.cc:3037] T 7a757b0c68de4143b74065852393fa13 P 7c852f00331741bf97c3d710d873c775 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:05.881887 26331 raft_consensus.cc:2454] T 7a757b0c68de4143b74065852393fa13 P 7c852f00331741bf97c3d710d873c775 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46cfab95e6e44c0be837c69674063c9 in term 1.
I20260619 02:51:05.881958 26040 raft_consensus.cc:2454] T 7a757b0c68de4143b74065852393fa13 P 10455eb59a7040a98cb6dc8988bc24b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46cfab95e6e44c0be837c69674063c9 in term 1.
I20260619 02:51:05.882220 26125 leader_election.cc:302] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c852f00331741bf97c3d710d873c775, f46cfab95e6e44c0be837c69674063c9; no voters:
I20260619 02:51:05.882493 26486 raft_consensus.cc:2781] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:51:05.882941 26486 raft_consensus.cc:686] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9 [term 1 LEADER]: Becoming Leader. State: Replica: f46cfab95e6e44c0be837c69674063c9, State: Running, Role: LEADER
I20260619 02:51:05.883344 26486 consensus_queue.cc:237] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46cfab95e6e44c0be837c69674063c9" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45195 } } peers { permanent_uuid: "10455eb59a7040a98cb6dc8988bc24b8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45679 } } peers { permanent_uuid: "7c852f00331741bf97c3d710d873c775" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34939 } }
I20260619 02:51:05.884155 26494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.883656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.18.134.2:45581)
0619 02:51:05.883879 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:05.883996 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260619 02:51:05.887171 25716 catalog_manager.cc:5495] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to f46cfab95e6e44c0be837c69674063c9 (127.18.134.2). New cstate: current_term: 1 leader_uuid: "f46cfab95e6e44c0be837c69674063c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f46cfab95e6e44c0be837c69674063c9" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10455eb59a7040a98cb6dc8988bc24b8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c852f00331741bf97c3d710d873c775" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34939 } health_report { overall_health: UNKNOWN } } }
I20260619 02:51:05.942601 26479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.937746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38204 (local address 127.18.134.2:45195)
0619 02:51:05.937918 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:05.937922 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:05.937938 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:51:05.938024 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:05.938028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:05.938099 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:05.938226 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:51:05.938528 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:05.939291 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:05.940581 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:05.941007 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:05.941105 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:05.941458 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:51:05.942014 (+ 556us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:51:05.942122 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:05.942273 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 02:51:05.942338 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 02:51:05.946439 26331 raft_consensus.cc:1261] T 7a757b0c68de4143b74065852393fa13 P 7c852f00331741bf97c3d710d873c775 [term 1 FOLLOWER]: Refusing update from remote peer f46cfab95e6e44c0be837c69674063c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:51:05.946812 26040 raft_consensus.cc:1261] T 7a757b0c68de4143b74065852393fa13 P 10455eb59a7040a98cb6dc8988bc24b8 [term 1 FOLLOWER]: Refusing update from remote peer f46cfab95e6e44c0be837c69674063c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:51:05.947119 26495 consensus_queue.cc:1035] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c852f00331741bf97c3d710d873c775" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:51:05.947522 26486 consensus_queue.cc:1035] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10455eb59a7040a98cb6dc8988bc24b8" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 45679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:51:05.949853 26466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.949369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.18.134.3:36385)
0619 02:51:05.949593 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:05.949702 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260619 02:51:05.949903 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.949413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.18.134.1:44987)
0619 02:51:05.949625 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:05.949716 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260619 02:51:05.954069 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.953630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.18.134.1:44257)
0619 02:51:05.953777 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:05.953878 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 02:51:05.964284 26494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.963778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.18.134.2:50097)
0619 02:51:05.963987 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:05.964079 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260619 02:51:05.968277 26494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.966767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.18.134.2:47509)
0619 02:51:05.967962 (+ 1195us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:05.968085 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1078}
I20260619 02:51:05.968302 26466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.967790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.18.134.3:34315)
0619 02:51:05.968028 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:05.968118 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260619 02:51:05.975224 25717 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:51:05.975498 25717 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:51:05.981191 25717 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a94a362219f40ec98932b3e1f58a9cd: Sending DeleteTablet for 3 replicas of tablet 7a757b0c68de4143b74065852393fa13
I20260619 02:51:05.981938 26170 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a757b0c68de4143b74065852393fa13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:51:05 UTC) from {username='oryx', principal='oryx/127.18.134.61@KRBTEST.COM'} at 127.0.0.1:38192
I20260619 02:51:05.982129 26311 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a757b0c68de4143b74065852393fa13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:51:05 UTC) from {username='oryx', principal='oryx/127.18.134.61@KRBTEST.COM'} at 127.0.0.1:55616
I20260619 02:51:05.982124 26020 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a757b0c68de4143b74065852393fa13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:51:05 UTC) from {username='oryx', principal='oryx/127.18.134.61@KRBTEST.COM'} at 127.0.0.1:49680
I20260619 02:51:05.982403 26493 tablet_replica.cc:330] T 7a757b0c68de4143b74065852393fa13 P 10455eb59a7040a98cb6dc8988bc24b8: stopping tablet replica
I20260619 02:51:05.982419 26492 tablet_replica.cc:330] T 7a757b0c68de4143b74065852393fa13 P 7c852f00331741bf97c3d710d873c775: stopping tablet replica
I20260619 02:51:05.982601 26492 raft_consensus.cc:2229] T 7a757b0c68de4143b74065852393fa13 P 7c852f00331741bf97c3d710d873c775 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:51:05.982616 26493 raft_consensus.cc:2229] T 7a757b0c68de4143b74065852393fa13 P 10455eb59a7040a98cb6dc8988bc24b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:51:05.982826 26492 raft_consensus.cc:2258] T 7a757b0c68de4143b74065852393fa13 P 7c852f00331741bf97c3d710d873c775 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:51:05.982868 26493 raft_consensus.cc:2258] T 7a757b0c68de4143b74065852393fa13 P 10455eb59a7040a98cb6dc8988bc24b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:51:05.983352 26508 tablet_replica.cc:330] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9: stopping tablet replica
I20260619 02:51:05.983579 26508 raft_consensus.cc:2229] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:51:05.984083 26493 ts_tablet_manager.cc:1905] T 7a757b0c68de4143b74065852393fa13 P 10455eb59a7040a98cb6dc8988bc24b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:51:05.984222 26508 raft_consensus.cc:2258] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:51:05.985172 26492 ts_tablet_manager.cc:1905] T 7a757b0c68de4143b74065852393fa13 P 7c852f00331741bf97c3d710d873c775: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:51:05.985299 26508 ts_tablet_manager.cc:1905] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:51:05.987151 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 25941
I20260619 02:51:05.990654 26493 ts_tablet_manager.cc:1918] T 7a757b0c68de4143b74065852393fa13 P 10455eb59a7040a98cb6dc8988bc24b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:51:05.990811 26493 log.cc:1196] T 7a757b0c68de4143b74065852393fa13 P 10455eb59a7040a98cb6dc8988bc24b8: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-0/wal/wals/7a757b0c68de4143b74065852393fa13
I20260619 02:51:05.991218 26493 ts_tablet_manager.cc:1939] T 7a757b0c68de4143b74065852393fa13 P 10455eb59a7040a98cb6dc8988bc24b8: Deleting consensus metadata
I20260619 02:51:05.992298 25703 catalog_manager.cc:4841] TS 10455eb59a7040a98cb6dc8988bc24b8 (127.18.134.1:45679): tablet 7a757b0c68de4143b74065852393fa13 (table test-table [id=adec3170d05d42ac8767aae4b47cc5d3]) successfully deleted
I20260619 02:51:05.994112 26492 ts_tablet_manager.cc:1918] T 7a757b0c68de4143b74065852393fa13 P 7c852f00331741bf97c3d710d873c775: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:51:05.994269 26492 log.cc:1196] T 7a757b0c68de4143b74065852393fa13 P 7c852f00331741bf97c3d710d873c775: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-2/wal/wals/7a757b0c68de4143b74065852393fa13
I20260619 02:51:05.994694 26492 ts_tablet_manager.cc:1939] T 7a757b0c68de4143b74065852393fa13 P 7c852f00331741bf97c3d710d873c775: Deleting consensus metadata
I20260619 02:51:05.995205 26508 ts_tablet_manager.cc:1918] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:51:05.995386 26508 log.cc:1196] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781837410515220-18968-0/minicluster-data/ts-1/wal/wals/7a757b0c68de4143b74065852393fa13
I20260619 02:51:05.995877 26508 ts_tablet_manager.cc:1939] T 7a757b0c68de4143b74065852393fa13 P f46cfab95e6e44c0be837c69674063c9: Deleting consensus metadata
I20260619 02:51:05.995877 25703 catalog_manager.cc:4841] TS 7c852f00331741bf97c3d710d873c775 (127.18.134.3:34939): tablet 7a757b0c68de4143b74065852393fa13 (table test-table [id=adec3170d05d42ac8767aae4b47cc5d3]) successfully deleted
I20260619 02:51:05.996910 25702 catalog_manager.cc:4841] TS f46cfab95e6e44c0be837c69674063c9 (127.18.134.2:45195): tablet 7a757b0c68de4143b74065852393fa13 (table test-table [id=adec3170d05d42ac8767aae4b47cc5d3]) successfully deleted
I20260619 02:51:06.006516 26480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:06.006096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.0.0.1:38458)
0619 02:51:06.006265 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:06.006359 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260619 02:51:06.007511 26480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:06.007129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33075 (local address 127.0.0.1:38474)
0619 02:51:06.007272 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:06.007366 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260619 02:51:06.009090 26481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:05.991997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49694 (local address 127.18.134.1:45679)
0619 02:51:05.992623 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:05.992628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:05.996632 (+ 4004us) server_negotiation.cc:407] Connection header received
0619 02:51:05.996917 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:05.996923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:05.997000 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:05.997089 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:05.998260 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:05.999070 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:06.000234 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:06.000621 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:06.000724 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:06.003217 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:06.003230 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:06.003245 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:06.003280 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:06.006095 (+ 2815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:06.006963 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:06.006972 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:06.006975 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:06.007126 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:06.007525 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:06.007528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:06.007531 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:06.007971 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:06.008248 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:06.008562 (+ 314us) server_negotiation.cc:299] Negotiation successful
0619 02:51:06.008761 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510}
I20260619 02:51:06.009665 26080 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:06.123256 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 25941
I20260619 02:51:06.140739 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 26097
I20260619 02:51:06.154754 26479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:06.141423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38218 (local address 127.18.134.2:45195)
0619 02:51:06.141604 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:06.141609 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:06.141721 (+ 112us) server_negotiation.cc:407] Connection header received
0619 02:51:06.141871 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:06.141875 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:06.141955 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:06.142032 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:06.143678 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:06.144433 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:06.145936 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:06.146408 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:06.146531 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:06.149142 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:06.149155 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:06.149170 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:06.149202 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:06.152103 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:06.152740 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:06.152747 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:06.152749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:06.152912 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:06.153271 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:06.153275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:06.153277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:06.153626 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:06.153985 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:06.154165 (+ 180us) server_negotiation.cc:299] Negotiation successful
0619 02:51:06.154379 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 02:51:06.155426 26230 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:06.271749 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 26097
I20260619 02:51:06.286506 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 26248
I20260619 02:51:06.299050 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:06.287245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55624 (local address 127.18.134.3:34939)
0619 02:51:06.287415 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:06.287419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:06.287436 (+ 17us) server_negotiation.cc:407] Connection header received
0619 02:51:06.287569 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:06.287572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:06.287633 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:06.287712 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:06.288977 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:06.289743 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:06.290937 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:06.291327 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:06.291425 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:06.293930 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:06.293944 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:06.293959 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:06.293995 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:06.296782 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:06.297383 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:06.297389 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:06.297392 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:06.297511 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:06.297808 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:06.297811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:06.297813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:06.298132 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:06.298327 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:06.298535 (+ 208us) server_negotiation.cc:299] Negotiation successful
0619 02:51:06.298727 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 02:51:06.299710 26371 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:06.389043 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 26248
I20260619 02:51:06.406266 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 25581
I20260619 02:51:06.407415 25747 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:06.524617 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 25581
I20260619 02:51:06.541638 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 25681
I20260619 02:51:06.542776 25934 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:06.651402 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 25681
2026-06-19T02:51:06Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9799 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:51:06 dist-test-slave-9mr0 krb5kdc[26518](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:51:06 dist-test-slave-9mr0 krb5kdc[26518](info): set up 2 sockets
Jun 19 02:51:06 dist-test-slave-9mr0 krb5kdc[26518](info): commencing operation
krb5kdc: starting...
W20260619 02:51:07.663285 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.973s user 0.000s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:51:07 dist-test-slave-9mr0 krb5kdc[26518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:51:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:51:07Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.62.keytab.
Entry for principal oryx/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.62.keytab.
I20260619 02:51:07.865896 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:37135
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.62.keytab
--principal=oryx/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:33575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:37135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:51:08.185971 26531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:08.186272 26531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:08.186384 26531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:08.193744 26531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:51:08.193851 26531 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 02:51:08.193894 26531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:08.193976 26531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:08.194036 26531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:51:08.194119 26531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:51:08.204130 26531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:33575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:37135
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.62.keytab
--principal=oryx/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:37135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:08.205770 26531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:08.207206 26531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:08.218600 26536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:08.218654 26539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:08.218936 26537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:08.220145 26531 server_base.cc:1034] running on GCE node
I20260619 02:51:08.220976 26531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:08.222393 26531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:08.223623 26531 hybrid_clock.cc:648] HybridClock initialized: now 1781837468223536 us; error 57 us; skew 500 ppm
I20260619 02:51:08.224011 26531 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:51:08 dist-test-slave-9mr0 krb5kdc[26518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837468, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:51:08.227242 26531 init.cc:377] Logged in from keytab as oryx/127.18.134.62@KRBTEST.COM (short username oryx)
I20260619 02:51:08.228729 26531 webserver.cc:460] Webserver started at http://127.18.134.62:34923/ using document root <none> and password file <none>
I20260619 02:51:08.229417 26531 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:08.229518 26531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:08.229830 26531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:51:08.232863 26531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "de7f10a123774146b91cb4077e5aebb1"
format_stamp: "Formatted at 2026-06-19 02:51:08 on dist-test-slave-9mr0"
server_key: "1371c178e446b4123c04a2d72323d45f"
server_key_iv: "aefb94e6b372debe2e2a9be5197ac3ac"
server_key_version: "encryptionkey@0"
I20260619 02:51:08.233631 26531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "de7f10a123774146b91cb4077e5aebb1"
format_stamp: "Formatted at 2026-06-19 02:51:08 on dist-test-slave-9mr0"
server_key: "1371c178e446b4123c04a2d72323d45f"
server_key_iv: "aefb94e6b372debe2e2a9be5197ac3ac"
server_key_version: "encryptionkey@0"
I20260619 02:51:08.238811 26531 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260619 02:51:08.242244 26546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:08.243737 26531 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:51:08.243901 26531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "de7f10a123774146b91cb4077e5aebb1"
format_stamp: "Formatted at 2026-06-19 02:51:08 on dist-test-slave-9mr0"
server_key: "1371c178e446b4123c04a2d72323d45f"
server_key_iv: "aefb94e6b372debe2e2a9be5197ac3ac"
server_key_version: "encryptionkey@0"
I20260619 02:51:08.244105 26531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:08.264415 26531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:08.268720 26531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:08.268973 26531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:08.289104 26531 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:37135
I20260619 02:51:08.289121 26607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:37135 every 8 connection(s)
I20260619 02:51:08.290409 26531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:51:08.294257 26608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:08.295387 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 26531
I20260619 02:51:08.295547 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:51:08.296139 18968 external_mini_cluster.cc:1428] Setting key 395beb52ce6c9e38162e88fd0909fe75
I20260619 02:51:08.308055 26608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de7f10a123774146b91cb4077e5aebb1: Bootstrap starting.
Jun 19 02:51:08 dist-test-slave-9mr0 krb5kdc[26518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM
I20260619 02:51:08.311847 26608 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de7f10a123774146b91cb4077e5aebb1: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:08.312919 26608 log.cc:824] T 00000000000000000000000000000000 P de7f10a123774146b91cb4077e5aebb1: Log is configured to *not* fsync() on all Append() calls
I20260619 02:51:08.316217 26608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de7f10a123774146b91cb4077e5aebb1: No bootstrap required, opened a new log
I20260619 02:51:08.319243 26611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:08.298027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38552 (local address 127.18.134.62:37135)
0619 02:51:08.298852 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:08.298864 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:08.298908 (+ 44us) server_negotiation.cc:407] Connection header received
0619 02:51:08.300417 (+ 1509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:08.300439 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:08.300866 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:08.301202 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:08.302934 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:08.304338 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:08.306090 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:08.306832 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:08.306966 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:08.311451 (+ 4485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:08.311460 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:08.311476 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:08.311514 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:08.314808 (+ 3294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:08.315719 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:08.315727 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:08.315732 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:08.315902 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:08.316354 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:08.316358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:08.316360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:08.316930 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:08.317140 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:08.317714 (+ 574us) server_negotiation.cc:299] Negotiation successful
0619 02:51:08.318156 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":334,"threads_started":1}
I20260619 02:51:08.324033 26608 raft_consensus.cc:348] T 00000000000000000000000000000000 P de7f10a123774146b91cb4077e5aebb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de7f10a123774146b91cb4077e5aebb1" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37135 } }
I20260619 02:51:08.324378 26608 raft_consensus.cc:374] T 00000000000000000000000000000000 P de7f10a123774146b91cb4077e5aebb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:08.324474 26608 raft_consensus.cc:729] T 00000000000000000000000000000000 P de7f10a123774146b91cb4077e5aebb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de7f10a123774146b91cb4077e5aebb1, State: Initialized, Role: FOLLOWER
I20260619 02:51:08.325048 26608 consensus_queue.cc:260] T 00000000000000000000000000000000 P de7f10a123774146b91cb4077e5aebb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de7f10a123774146b91cb4077e5aebb1" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37135 } }
I20260619 02:51:08.325263 26608 raft_consensus.cc:388] T 00000000000000000000000000000000 P de7f10a123774146b91cb4077e5aebb1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:51:08.325387 26608 raft_consensus.cc:482] T 00000000000000000000000000000000 P de7f10a123774146b91cb4077e5aebb1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:51:08.325537 26608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de7f10a123774146b91cb4077e5aebb1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:08.327723 26608 raft_consensus.cc:504] T 00000000000000000000000000000000 P de7f10a123774146b91cb4077e5aebb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de7f10a123774146b91cb4077e5aebb1" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37135 } }
I20260619 02:51:08.328222 26608 leader_election.cc:302] T 00000000000000000000000000000000 P de7f10a123774146b91cb4077e5aebb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de7f10a123774146b91cb4077e5aebb1; no voters:
I20260619 02:51:08.328756 26608 leader_election.cc:288] T 00000000000000000000000000000000 P de7f10a123774146b91cb4077e5aebb1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:51:08.328867 26613 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de7f10a123774146b91cb4077e5aebb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:51:08.329463 26613 raft_consensus.cc:686] T 00000000000000000000000000000000 P de7f10a123774146b91cb4077e5aebb1 [term 1 LEADER]: Becoming Leader. State: Replica: de7f10a123774146b91cb4077e5aebb1, State: Running, Role: LEADER
I20260619 02:51:08.329984 26613 consensus_queue.cc:237] T 00000000000000000000000000000000 P de7f10a123774146b91cb4077e5aebb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de7f10a123774146b91cb4077e5aebb1" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37135 } }
I20260619 02:51:08.330729 26608 sys_catalog.cc:564] T 00000000000000000000000000000000 P de7f10a123774146b91cb4077e5aebb1 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:51:08.332442 26614 sys_catalog.cc:455] T 00000000000000000000000000000000 P de7f10a123774146b91cb4077e5aebb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de7f10a123774146b91cb4077e5aebb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de7f10a123774146b91cb4077e5aebb1" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37135 } } }
I20260619 02:51:08.332885 26614 sys_catalog.cc:458] T 00000000000000000000000000000000 P de7f10a123774146b91cb4077e5aebb1 [sys.catalog]: This master's current role is: LEADER
I20260619 02:51:08.332811 26615 sys_catalog.cc:455] T 00000000000000000000000000000000 P de7f10a123774146b91cb4077e5aebb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de7f10a123774146b91cb4077e5aebb1. Latest consensus state: current_term: 1 leader_uuid: "de7f10a123774146b91cb4077e5aebb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de7f10a123774146b91cb4077e5aebb1" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37135 } } }
I20260619 02:51:08.333137 26615 sys_catalog.cc:458] T 00000000000000000000000000000000 P de7f10a123774146b91cb4077e5aebb1 [sys.catalog]: This master's current role is: LEADER
I20260619 02:51:08.333869 26621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:51:08.338919 26621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:51:08.348804 26621 catalog_manager.cc:1347] Generated new cluster ID: a9f497e420b94cbca8651fef228c6da9
I20260619 02:51:08.348929 26621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:51:08.359740 26621 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:51:08.361017 26621 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:51:08.368008 26621 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de7f10a123774146b91cb4077e5aebb1: Generated new TSK 0
I20260619 02:51:08.368956 26621 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.134.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.134.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.134.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.1.keytab.
Entry for principal oryx/127.18.134.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.1.keytab.
I20260619 02:51:08.430173 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:0
--local_ip_for_outbound_sockets=127.18.134.1
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:37135
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.1.keytab
--principal=oryx/127.18.134.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:33575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:51:08.742910 26634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:08.743227 26634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:08.743362 26634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:08.750581 26634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:08.750720 26634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:08.750818 26634 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:51:08.750895 26634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:51:08.760525 26634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:33575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.1.keytab
--principal=oryx/127.18.134.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.134.62:37135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:08.761736 26634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:08.763028 26634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:08.776945 26640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:08.777164 26639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:08.777464 26642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:08.778245 26634 server_base.cc:1034] running on GCE node
I20260619 02:51:08.778957 26634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:08.779875 26634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:08.781106 26634 hybrid_clock.cc:648] HybridClock initialized: now 1781837468781034 us; error 44 us; skew 500 ppm
I20260619 02:51:08.781411 26634 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:51:08 dist-test-slave-9mr0 krb5kdc[26518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837468, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:51:08.789716 26634 init.cc:377] Logged in from keytab as oryx/127.18.134.1@KRBTEST.COM (short username oryx)
I20260619 02:51:08.791167 26634 webserver.cc:460] Webserver started at http://127.18.134.1:40773/ using document root <none> and password file <none>
I20260619 02:51:08.791824 26634 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:08.791930 26634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:08.792219 26634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:51:08.795138 26634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-0/data/instance:
uuid: "a1d869606d534bd488f4f667e355a57a"
format_stamp: "Formatted at 2026-06-19 02:51:08 on dist-test-slave-9mr0"
server_key: "4776ba3e467449584714035dde90109b"
server_key_iv: "8759a9a2b6aa103f86cdd3f1e953ef29"
server_key_version: "encryptionkey@0"
I20260619 02:51:08.795850 26634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance:
uuid: "a1d869606d534bd488f4f667e355a57a"
format_stamp: "Formatted at 2026-06-19 02:51:08 on dist-test-slave-9mr0"
server_key: "4776ba3e467449584714035dde90109b"
server_key_iv: "8759a9a2b6aa103f86cdd3f1e953ef29"
server_key_version: "encryptionkey@0"
I20260619 02:51:08.800885 26634 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:51:08.804294 26649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:08.805637 26634 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 02:51:08.805781 26634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "a1d869606d534bd488f4f667e355a57a"
format_stamp: "Formatted at 2026-06-19 02:51:08 on dist-test-slave-9mr0"
server_key: "4776ba3e467449584714035dde90109b"
server_key_iv: "8759a9a2b6aa103f86cdd3f1e953ef29"
server_key_version: "encryptionkey@0"
I20260619 02:51:08.805972 26634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:08.821484 26634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:08.825465 26634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:08.825708 26634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:08.828513 26634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:51:08.828610 26634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:08.828766 26634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:51:08.828876 26634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Jun 19 02:51:08 dist-test-slave-9mr0 krb5kdc[26518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837468, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM
I20260619 02:51:08.850276 26657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:08.828934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:59201 (local address 127.18.134.62:37135)
0619 02:51:08.829414 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:08.829420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:08.830715 (+ 1295us) server_negotiation.cc:407] Connection header received
0619 02:51:08.833093 (+ 2378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:08.833100 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:08.833195 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:08.833317 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:08.834771 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:08.835888 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:08.838246 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:08.838822 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:08.838945 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:08.843279 (+ 4334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:08.843286 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:08.843291 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:08.843332 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:08.846662 (+ 3330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:08.847590 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:08.847598 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:08.847602 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:08.847776 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:08.848400 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:08.848405 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:08.848409 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:08.848792 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:08.848951 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:08.849639 (+ 688us) server_negotiation.cc:299] Negotiation successful
0619 02:51:08.849872 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":245,"threads_started":1}
I20260619 02:51:08.851209 26659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:08.829647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:37135 (local address 127.18.134.1:59201)
0619 02:51:08.830541 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:08.830590 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:08.832802 (+ 2212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:08.833542 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:08.833551 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:08.834032 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:08.834591 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:08.834607 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:08.836117 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:08.836123 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:08.838030 (+ 1907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:08.838045 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:08.839154 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:08.839163 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:08.839485 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:08.840283 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:08.840305 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:08.843031 (+ 2726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:08.846811 (+ 3780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:08.846820 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:08.846841 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:08.847442 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:08.847907 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:08.847911 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:08.847914 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:08.848225 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:08.848926 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:08.848933 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:08.849281 (+ 348us) client_negotiation.cc:769] Sending connection context
0619 02:51:08.849504 (+ 223us) client_negotiation.cc:240] Negotiation successful
0619 02:51:08.849893 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":352,"threads_started":1}
I20260619 02:51:08.868726 26634 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:33759
I20260619 02:51:08.868774 26769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:33759 every 8 connection(s)
I20260619 02:51:08.869961 26634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:51:08.871984 26770 heartbeater.cc:347] Connected to a master server at 127.18.134.62:37135
I20260619 02:51:08.872534 26770 heartbeater.cc:464] Registering TS with master...
I20260619 02:51:08.873188 26770 heartbeater.cc:511] Master 127.18.134.62:37135 requested a full tablet report, sending...
I20260619 02:51:08.874502 26657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:08.860178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:41443 (local address 127.18.134.62:37135)
0619 02:51:08.860370 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:08.860376 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:08.860971 (+ 595us) server_negotiation.cc:407] Connection header received
0619 02:51:08.861195 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:08.861201 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:08.861279 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:08.861349 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:08.862910 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:08.864003 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:08.866213 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:08.866852 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:08.866962 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:08.869143 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:08.869149 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:08.869153 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:08.869184 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:08.872073 (+ 2889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:08.872715 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:08.872721 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:08.872725 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:08.872846 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:08.873174 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:08.873179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:08.873183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:08.873431 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:08.873553 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:08.873934 (+ 381us) server_negotiation.cc:299] Negotiation successful
0619 02:51:08.874148 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 02:51:08.874547 26745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:08.860235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:37135 (local address 127.18.134.1:41443)
0619 02:51:08.860872 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:08.860890 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:08.861028 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:08.861566 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:08.861571 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:08.862215 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:08.862725 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:08.862739 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:08.864223 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:08.864231 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:08.866001 (+ 1770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:08.866020 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:08.867024 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:08.867030 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:08.867183 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:08.867800 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:08.867825 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:08.868964 (+ 1139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:08.872192 (+ 3228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:08.872198 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:08.872201 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:08.872612 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:08.872944 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:08.872949 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:08.872957 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:08.873078 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:08.873535 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:08.873540 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:08.873718 (+ 178us) client_negotiation.cc:769] Sending connection context
0619 02:51:08.873834 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 02:51:08.874133 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":358,"threads_started":1}
I20260619 02:51:08.875152 26563 ts_manager.cc:194] Registered new tserver with Master: a1d869606d534bd488f4f667e355a57a (127.18.134.1:33759)
I20260619 02:51:08.876519 26563 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.134.1@KRBTEST.COM'} at 127.18.134.1:59201
I20260619 02:51:08.879281 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 26634
I20260619 02:51:08.879460 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance
I20260619 02:51:08.880097 18968 external_mini_cluster.cc:1428] Setting key 6d5c90146c5e63726d3e2977f4ba3ab1
WARNING: no policy specified for oryx/127.18.134.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.134.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.134.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.2.keytab.
Entry for principal oryx/127.18.134.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.2.keytab.
I20260619 02:51:08.923408 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:0
--local_ip_for_outbound_sockets=127.18.134.2
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:37135
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.2.keytab
--principal=oryx/127.18.134.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:33575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:51:09.234776 26774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:09.235055 26774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:09.235179 26774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:09.242353 26774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:09.242466 26774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:09.242574 26774 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:51:09.242703 26774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:51:09.252460 26774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:33575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.2.keytab
--principal=oryx/127.18.134.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.134.62:37135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:09.253782 26774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:09.255174 26774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:09.267724 26780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:09.267689 26782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:09.267894 26779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:09.268505 26774 server_base.cc:1034] running on GCE node
I20260619 02:51:09.269116 26774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:09.270056 26774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:09.271297 26774 hybrid_clock.cc:648] HybridClock initialized: now 1781837469271233 us; error 44 us; skew 500 ppm
I20260619 02:51:09.271653 26774 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:51:09 dist-test-slave-9mr0 krb5kdc[26518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837469, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:51:09.275087 26774 init.cc:377] Logged in from keytab as oryx/127.18.134.2@KRBTEST.COM (short username oryx)
I20260619 02:51:09.276641 26774 webserver.cc:460] Webserver started at http://127.18.134.2:45519/ using document root <none> and password file <none>
I20260619 02:51:09.277330 26774 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:09.277432 26774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:09.277738 26774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:51:09.280716 26774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-1/data/instance:
uuid: "6a6db57534424adabd9ce75da8a7c4b3"
format_stamp: "Formatted at 2026-06-19 02:51:09 on dist-test-slave-9mr0"
server_key: "d9f1cd99634b2271a79a60f941e46b42"
server_key_iv: "178e61461f92abd77d7184d28785379c"
server_key_version: "encryptionkey@0"
I20260619 02:51:09.281498 26774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance:
uuid: "6a6db57534424adabd9ce75da8a7c4b3"
format_stamp: "Formatted at 2026-06-19 02:51:09 on dist-test-slave-9mr0"
server_key: "d9f1cd99634b2271a79a60f941e46b42"
server_key_iv: "178e61461f92abd77d7184d28785379c"
server_key_version: "encryptionkey@0"
I20260619 02:51:09.286701 26774 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 02:51:09.290359 26789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:09.291857 26774 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:51:09.292021 26774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "6a6db57534424adabd9ce75da8a7c4b3"
format_stamp: "Formatted at 2026-06-19 02:51:09 on dist-test-slave-9mr0"
server_key: "d9f1cd99634b2271a79a60f941e46b42"
server_key_iv: "178e61461f92abd77d7184d28785379c"
server_key_version: "encryptionkey@0"
I20260619 02:51:09.292233 26774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:09.312165 26774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:09.316107 26774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:09.316349 26774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:09.318986 26774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:51:09.319080 26774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:09.319231 26774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:51:09.319305 26774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:51:09 dist-test-slave-9mr0 krb5kdc[26518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837469, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM
I20260619 02:51:09.341470 26657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:09.319481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:55829 (local address 127.18.134.62:37135)
0619 02:51:09.319679 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:09.319685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:09.321187 (+ 1502us) server_negotiation.cc:407] Connection header received
0619 02:51:09.323230 (+ 2043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:09.323236 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:09.323317 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:09.323435 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:09.325384 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:09.326374 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:09.329275 (+ 2901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:09.329931 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:09.330069 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:09.334593 (+ 4524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:09.334599 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:09.334605 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:09.334678 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:09.337664 (+ 2986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:09.338873 (+ 1209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:09.338879 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:09.338883 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:09.339020 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:09.339672 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:09.339676 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:09.339678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:09.339952 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:09.340078 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:09.341049 (+ 971us) server_negotiation.cc:299] Negotiation successful
0619 02:51:09.341224 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 02:51:09.343174 26798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:09.320154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:37135 (local address 127.18.134.2:55829)
0619 02:51:09.321000 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:09.321048 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:09.322910 (+ 1862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:09.323670 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:09.323679 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:09.324343 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:09.325181 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:09.325214 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:09.326554 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:09.326560 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:09.329100 (+ 2540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:09.329118 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:09.330198 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:09.330206 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:09.330459 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:09.331224 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:09.331265 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:09.334380 (+ 3115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:09.337820 (+ 3440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:09.338116 (+ 296us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:09.338136 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:09.338715 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:09.339148 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:09.339154 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:09.339158 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:09.339499 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:09.340084 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:09.340094 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:09.340594 (+ 500us) client_negotiation.cc:769] Sending connection context
0619 02:51:09.340920 (+ 326us) client_negotiation.cc:240] Negotiation successful
0619 02:51:09.341510 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":357,"threads_started":1}
I20260619 02:51:09.363157 26868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:09.349226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:37135 (local address 127.18.134.2:43721)
0619 02:51:09.349808 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:09.349828 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:09.349968 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:09.350302 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:09.350306 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:09.350869 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:09.351286 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:09.351299 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:09.352471 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:09.352477 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:09.353499 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:09.353512 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:09.354245 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:09.354252 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:09.354412 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:09.355034 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:09.355058 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:09.356155 (+ 1097us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:09.359967 (+ 3812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:09.359978 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:09.359982 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:09.360511 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:09.360936 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:09.360941 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:09.360943 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:09.361091 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:09.361796 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:09.361801 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:09.362026 (+ 225us) client_negotiation.cc:769] Sending connection context
0619 02:51:09.362227 (+ 201us) client_negotiation.cc:240] Negotiation successful
0619 02:51:09.362655 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":452,"threads_started":1}
I20260619 02:51:09.363907 26657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:09.349641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:43721 (local address 127.18.134.62:37135)
0619 02:51:09.349842 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:09.349849 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:09.349894 (+ 45us) server_negotiation.cc:407] Connection header received
0619 02:51:09.350081 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:09.350088 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:09.350171 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:09.350285 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:09.351419 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:09.352347 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:09.353649 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:09.354069 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:09.354182 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:09.356378 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:09.356386 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:09.356392 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:09.356434 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:09.359823 (+ 3389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:09.360634 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:09.360640 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:09.360643 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:09.360807 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:09.361201 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:09.361206 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:09.361209 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:09.361645 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:09.361804 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:09.363477 (+ 1673us) server_negotiation.cc:299] Negotiation successful
0619 02:51:09.363713 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:51:09.371523 26774 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:33961
I20260619 02:51:09.371551 26908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:33961 every 8 connection(s)
I20260619 02:51:09.372725 26774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:51:09.374679 26909 heartbeater.cc:347] Connected to a master server at 127.18.134.62:37135
I20260619 02:51:09.374912 26909 heartbeater.cc:464] Registering TS with master...
I20260619 02:51:09.375521 26909 heartbeater.cc:511] Master 127.18.134.62:37135 requested a full tablet report, sending...
I20260619 02:51:09.376760 26563 ts_manager.cc:194] Registered new tserver with Master: 6a6db57534424adabd9ce75da8a7c4b3 (127.18.134.2:33961)
I20260619 02:51:09.377516 26563 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.134.2@KRBTEST.COM'} at 127.18.134.2:55829
I20260619 02:51:09.382768 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 26774
I20260619 02:51:09.382926 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance
I20260619 02:51:09.383364 18968 external_mini_cluster.cc:1428] Setting key f3dbe7b34961085b8db04ad36bce4168
WARNING: no policy specified for oryx/127.18.134.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.134.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.134.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.3.keytab.
Entry for principal oryx/127.18.134.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.3.keytab.
I20260619 02:51:09.425791 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:0
--local_ip_for_outbound_sockets=127.18.134.3
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:37135
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.3.keytab
--principal=oryx/127.18.134.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:33575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:51:09.737133 26913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:09.737426 26913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:09.737553 26913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:09.745044 26913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:09.745162 26913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:09.745271 26913 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 02:51:09.745358 26913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:51:09.755375 26913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:33575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.3.keytab
--principal=oryx/127.18.134.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.134.62:37135
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:09.756639 26913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:09.757898 26913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:09.771239 26921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:09.771404 26918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:09.771404 26919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:09.772850 26913 server_base.cc:1034] running on GCE node
I20260619 02:51:09.773487 26913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:09.774407 26913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:09.775636 26913 hybrid_clock.cc:648] HybridClock initialized: now 1781837469775550 us; error 67 us; skew 500 ppm
I20260619 02:51:09.775943 26913 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:51:09 dist-test-slave-9mr0 krb5kdc[26518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837469, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:51:09.779094 26913 init.cc:377] Logged in from keytab as oryx/127.18.134.3@KRBTEST.COM (short username oryx)
I20260619 02:51:09.780580 26913 webserver.cc:460] Webserver started at http://127.18.134.3:37237/ using document root <none> and password file <none>
I20260619 02:51:09.781267 26913 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:09.781368 26913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:09.781670 26913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:51:09.784612 26913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-2/data/instance:
uuid: "a88e92ebea134aecbf7fd8de8e29dae9"
format_stamp: "Formatted at 2026-06-19 02:51:09 on dist-test-slave-9mr0"
server_key: "f64d330e5de838941a7025bbd255aa08"
server_key_iv: "57a46acc032eadb50b3fd79c224992a9"
server_key_version: "encryptionkey@0"
I20260619 02:51:09.785378 26913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance:
uuid: "a88e92ebea134aecbf7fd8de8e29dae9"
format_stamp: "Formatted at 2026-06-19 02:51:09 on dist-test-slave-9mr0"
server_key: "f64d330e5de838941a7025bbd255aa08"
server_key_iv: "57a46acc032eadb50b3fd79c224992a9"
server_key_version: "encryptionkey@0"
I20260619 02:51:09.791610 26913 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260619 02:51:09.795529 26928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:09.797109 26913 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260619 02:51:09.797279 26913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "a88e92ebea134aecbf7fd8de8e29dae9"
format_stamp: "Formatted at 2026-06-19 02:51:09 on dist-test-slave-9mr0"
server_key: "f64d330e5de838941a7025bbd255aa08"
server_key_iv: "57a46acc032eadb50b3fd79c224992a9"
server_key_version: "encryptionkey@0"
I20260619 02:51:09.797492 26913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:09.831228 26913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:09.835763 26913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:09.836035 26913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:09.838817 26913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:51:09.838918 26913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:09.839078 26913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:51:09.839149 26913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:51:09 dist-test-slave-9mr0 krb5kdc[26518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837469, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM
I20260619 02:51:09.863911 26657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:09.839763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:56539 (local address 127.18.134.62:37135)
0619 02:51:09.839920 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:09.839924 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:09.841560 (+ 1636us) server_negotiation.cc:407] Connection header received
0619 02:51:09.843969 (+ 2409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:09.843975 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:09.844063 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:09.844163 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:09.847099 (+ 2936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:09.848292 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:09.850944 (+ 2652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:09.851621 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:09.851718 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:09.856758 (+ 5040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:09.856764 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:09.856767 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:09.856799 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:09.859942 (+ 3143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:09.861124 (+ 1182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:09.861131 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:09.861134 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:09.861303 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:09.861856 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:09.861861 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:09.861866 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:09.862248 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:09.862417 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:09.863343 (+ 926us) server_negotiation.cc:299] Negotiation successful
0619 02:51:09.863576 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 02:51:09.865038 26937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:09.840569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:37135 (local address 127.18.134.3:56539)
0619 02:51:09.841395 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:09.841436 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:09.843625 (+ 2189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:09.844616 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:09.844626 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:09.845751 (+ 1125us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:09.846830 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:09.846863 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:09.848524 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:09.848530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:09.850763 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:09.850779 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:09.852081 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:09.852088 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:09.852382 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:09.853124 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:09.853156 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:09.856451 (+ 3295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:09.860118 (+ 3667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:09.860132 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:09.860152 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:09.860945 (+ 793us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:09.861408 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:09.861413 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:09.861418 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:09.861706 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:09.862389 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:09.862401 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:09.862911 (+ 510us) client_negotiation.cc:769] Sending connection context
0619 02:51:09.863195 (+ 284us) client_negotiation.cc:240] Negotiation successful
0619 02:51:09.863738 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":365,"threads_started":1}
I20260619 02:51:09.879365 26770 heartbeater.cc:503] Master 127.18.134.62:37135 was elected leader, sending a full tablet report...
I20260619 02:51:09.884433 26992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:09.870984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:37135 (local address 127.18.134.3:49773)
0619 02:51:09.871619 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:09.871638 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:09.871775 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:09.872154 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:09.872159 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:09.872675 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:09.873007 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:09.873018 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:09.874483 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:09.874489 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:09.875774 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:09.875786 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:09.876805 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:09.876810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:09.876956 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:09.877642 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:09.877665 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:09.878425 (+ 760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:09.881924 (+ 3499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:09.881929 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:09.881933 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:09.882307 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:09.882684 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:09.882688 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:09.882690 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:09.882810 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:09.883449 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:09.883452 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:09.883662 (+ 210us) client_negotiation.cc:769] Sending connection context
0619 02:51:09.883823 (+ 161us) client_negotiation.cc:240] Negotiation successful
0619 02:51:09.884102 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":329,"threads_started":1}
I20260619 02:51:09.884819 26657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:09.870901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:49773 (local address 127.18.134.62:37135)
0619 02:51:09.871081 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:09.871087 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:09.871740 (+ 653us) server_negotiation.cc:407] Connection header received
0619 02:51:09.871901 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:09.871907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:09.872006 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:09.872114 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:09.873161 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:09.874311 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:09.875982 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:09.876605 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:09.876762 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:09.878606 (+ 1844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:09.878613 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:09.878618 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:09.878688 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:09.881802 (+ 3114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:09.882422 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:09.882427 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:09.882432 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:09.882559 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:09.882907 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:09.882913 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:09.882916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:09.883258 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:09.883397 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:09.883981 (+ 584us) server_negotiation.cc:299] Negotiation successful
0619 02:51:09.884237 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 02:51:09.899061 26913 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:33887
I20260619 02:51:09.899089 27047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:33887 every 8 connection(s)
I20260619 02:51:09.900262 26913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:51:09.902320 27048 heartbeater.cc:347] Connected to a master server at 127.18.134.62:37135
I20260619 02:51:09.902511 27048 heartbeater.cc:464] Registering TS with master...
I20260619 02:51:09.903172 27048 heartbeater.cc:511] Master 127.18.134.62:37135 requested a full tablet report, sending...
I20260619 02:51:09.904383 26563 ts_manager.cc:194] Registered new tserver with Master: a88e92ebea134aecbf7fd8de8e29dae9 (127.18.134.3:33887)
I20260619 02:51:09.905303 26563 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.134.3@KRBTEST.COM'} at 127.18.134.3:56539
I20260619 02:51:09.905385 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 26913
I20260619 02:51:09.905534 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance
I20260619 02:51:09.906049 18968 external_mini_cluster.cc:1428] Setting key dc67192477c212be305a0f91f87f8022
I20260619 02:51:09.908854 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 02:51:09 dist-test-slave-9mr0 krb5kdc[26518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837469, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 02:51:09 dist-test-slave-9mr0 krb5kdc[26518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:09 dist-test-slave-9mr0 krb5kdc[26518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:10.379930 26909 heartbeater.cc:503] Master 127.18.134.62:37135 was elected leader, sending a full tablet report...
I20260619 02:51:10.908093 27048 heartbeater.cc:503] Master 127.18.134.62:37135 was elected leader, sending a full tablet report...
W20260619 02:51:10.919178 27056 sasl_common.cc:323] Time spent calling sasl_client_start(): real 0.970s user 0.001s sys 0.013s
W20260619 02:51:10.920287 26657 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:09.943719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36716 (local address 127.18.134.62:37135)
0619 02:51:09.943946 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:09.943953 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:09.944172 (+ 219us) server_negotiation.cc:407] Connection header received
0619 02:51:09.944373 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:09.944379 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:09.944464 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:09.944574 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:09.945876 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:09.946623 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:09.947920 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:09.948347 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:09.948458 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:10.919969 (+971511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36716: BlockingRecv error: recv got EOF from 127.0.0.1:36716 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Jun 19 02:51:10 dist-test-slave-9mr0 krb5kdc[26518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837469, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM
I20260619 02:51:10.937420 26657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:10.924292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36718 (local address 127.18.134.62:37135)
0619 02:51:10.924505 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:10.924511 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:10.924814 (+ 303us) server_negotiation.cc:407] Connection header received
0619 02:51:10.924984 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:10.924989 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:10.925074 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:10.925168 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:10.926557 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:10.927308 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:10.928553 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:10.928957 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:10.929067 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:10.932813 (+ 3746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:10.932818 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:10.932822 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:10.932858 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:10.935253 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:10.935884 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:10.935892 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:10.935894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:10.936009 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:10.936330 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:10.936333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:10.936335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:10.936621 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:10.936846 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:10.937023 (+ 177us) server_negotiation.cc:299] Negotiation successful
0619 02:51:10.937188 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 02:51:10.941511 26563 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:51:10.948573 26563 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:51:10.970520 27069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:10.960505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48246 (local address 127.18.134.2:33961)
0619 02:51:10.962377 (+ 1872us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:10.962384 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:10.962411 (+ 27us) server_negotiation.cc:407] Connection header received
0619 02:51:10.962506 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:10.962515 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:10.962802 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:10.963594 (+ 792us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:10.963786 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:10.964700 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:10.967631 (+ 2931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:10.969089 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:10.969225 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:10.969437 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:10.970047 (+ 610us) server_negotiation.cc:299] Negotiation successful
0619 02:51:10.970229 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1738,"thread_start_us":1599,"threads_started":1}
I20260619 02:51:10.970592 27068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:10.960311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:33961 (local address 127.0.0.1:48246)
0619 02:51:10.961292 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:10.961335 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:10.961519 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:10.963031 (+ 1512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:10.963038 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:10.963065 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:10.963556 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:10.963568 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:10.964824 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:10.964829 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:10.967431 (+ 2602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:10.967446 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:10.969591 (+ 2145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:10.969599 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:10.969785 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:10.969841 (+ 56us) client_negotiation.cc:769] Sending connection context
0619 02:51:10.969984 (+ 143us) client_negotiation.cc:240] Negotiation successful
0619 02:51:10.970175 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"mutex_wait_us":1,"thread_start_us":587,"threads_started":1}
I20260619 02:51:10.972404 27067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:10.960402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:33887 (local address 127.0.0.1:57604)
0619 02:51:10.962445 (+ 2043us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:10.962460 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:10.962671 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:10.963473 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:10.963481 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:10.963507 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:10.965895 (+ 2388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:10.965906 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:10.967696 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:10.967701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:10.969663 (+ 1962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:10.969676 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:10.971551 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:10.971557 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:10.971721 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:10.971754 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 02:51:10.971858 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 02:51:10.972006 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1943,"mutex_wait_us":1980,"thread_start_us":1814,"threads_started":1}
I20260619 02:51:10.972407 27070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:10.960879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57604 (local address 127.18.134.3:33887)
0619 02:51:10.962064 (+ 1185us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:10.962073 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:10.962541 (+ 468us) server_negotiation.cc:407] Connection header received
0619 02:51:10.962846 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:10.962854 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:10.963068 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:10.963227 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:10.966077 (+ 2850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:10.967540 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:10.969856 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:10.971331 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:10.971478 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:10.971689 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:10.971922 (+ 233us) server_negotiation.cc:299] Negotiation successful
0619 02:51:10.972103 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1048,"thread_start_us":936,"threads_started":1}
I20260619 02:51:10.974895 26839 tablet_service.cc:1467] Processing CreateTablet for tablet 7f65771b15a7462a8574404f3b450af0 (DEFAULT_TABLE table=test-table [id=0aa87dcec312428d879b6e903133919b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:51:10.975835 27071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:10.962590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:33759 (local address 127.0.0.1:34020)
0619 02:51:10.965280 (+ 2690us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:10.965297 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:10.965398 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:10.968690 (+ 3292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:10.968695 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:10.968720 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:10.969113 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:10.969125 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:10.971100 (+ 1975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:10.971105 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:10.973036 (+ 1931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:10.973051 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:10.975068 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:10.975074 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:10.975246 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:10.975277 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 02:51:10.975373 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 02:51:10.975505 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2572,"mutex_wait_us":4,"thread_start_us":2421,"threads_started":1}
I20260619 02:51:10.975857 27072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:10.962908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34020 (local address 127.18.134.1:33759)
0619 02:51:10.968067 (+ 5159us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:10.968076 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:10.968104 (+ 28us) server_negotiation.cc:407] Connection header received
0619 02:51:10.968208 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:10.968216 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:10.968456 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:10.969337 (+ 881us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:10.969513 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:10.970920 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:10.973187 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:10.974866 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:10.975005 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:10.975206 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:10.975427 (+ 221us) server_negotiation.cc:299] Negotiation successful
0619 02:51:10.975582 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5005,"thread_start_us":4856,"threads_started":1}
I20260619 02:51:10.976564 26839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f65771b15a7462a8574404f3b450af0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:10.976449 26978 tablet_service.cc:1467] Processing CreateTablet for tablet 7f65771b15a7462a8574404f3b450af0 (DEFAULT_TABLE table=test-table [id=0aa87dcec312428d879b6e903133919b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:51:10.977726 26978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f65771b15a7462a8574404f3b450af0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:10.979557 26700 tablet_service.cc:1467] Processing CreateTablet for tablet 7f65771b15a7462a8574404f3b450af0 (DEFAULT_TABLE table=test-table [id=0aa87dcec312428d879b6e903133919b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:51:10.980846 26700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f65771b15a7462a8574404f3b450af0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:10.990367 27074 tablet_bootstrap.cc:492] T 7f65771b15a7462a8574404f3b450af0 P 6a6db57534424adabd9ce75da8a7c4b3: Bootstrap starting.
I20260619 02:51:10.990788 27073 tablet_bootstrap.cc:492] T 7f65771b15a7462a8574404f3b450af0 P a88e92ebea134aecbf7fd8de8e29dae9: Bootstrap starting.
I20260619 02:51:10.994093 27074 tablet_bootstrap.cc:654] T 7f65771b15a7462a8574404f3b450af0 P 6a6db57534424adabd9ce75da8a7c4b3: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:10.994093 27073 tablet_bootstrap.cc:654] T 7f65771b15a7462a8574404f3b450af0 P a88e92ebea134aecbf7fd8de8e29dae9: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:10.994371 27075 tablet_bootstrap.cc:492] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a: Bootstrap starting.
I20260619 02:51:10.995481 27074 log.cc:824] T 7f65771b15a7462a8574404f3b450af0 P 6a6db57534424adabd9ce75da8a7c4b3: Log is configured to *not* fsync() on all Append() calls
I20260619 02:51:10.995712 27073 log.cc:824] T 7f65771b15a7462a8574404f3b450af0 P a88e92ebea134aecbf7fd8de8e29dae9: Log is configured to *not* fsync() on all Append() calls
I20260619 02:51:10.997640 27075 tablet_bootstrap.cc:654] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:10.998543 27075 log.cc:824] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:51:10.998589 27073 tablet_bootstrap.cc:492] T 7f65771b15a7462a8574404f3b450af0 P a88e92ebea134aecbf7fd8de8e29dae9: No bootstrap required, opened a new log
I20260619 02:51:10.998837 27073 ts_tablet_manager.cc:1397] T 7f65771b15a7462a8574404f3b450af0 P a88e92ebea134aecbf7fd8de8e29dae9: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260619 02:51:11.001155 27075 tablet_bootstrap.cc:492] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a: No bootstrap required, opened a new log
I20260619 02:51:11.001163 27074 tablet_bootstrap.cc:492] T 7f65771b15a7462a8574404f3b450af0 P 6a6db57534424adabd9ce75da8a7c4b3: No bootstrap required, opened a new log
I20260619 02:51:11.001410 27075 ts_tablet_manager.cc:1397] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260619 02:51:11.001410 27074 ts_tablet_manager.cc:1397] T 7f65771b15a7462a8574404f3b450af0 P 6a6db57534424adabd9ce75da8a7c4b3: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260619 02:51:11.006043 27073 raft_consensus.cc:348] T 7f65771b15a7462a8574404f3b450af0 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } } peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } }
I20260619 02:51:11.006390 27073 raft_consensus.cc:374] T 7f65771b15a7462a8574404f3b450af0 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:11.006495 27073 raft_consensus.cc:729] T 7f65771b15a7462a8574404f3b450af0 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a88e92ebea134aecbf7fd8de8e29dae9, State: Initialized, Role: FOLLOWER
I20260619 02:51:11.007265 27073 consensus_queue.cc:260] T 7f65771b15a7462a8574404f3b450af0 P a88e92ebea134aecbf7fd8de8e29dae9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } } peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } }
I20260619 02:51:11.009694 27073 ts_tablet_manager.cc:1428] T 7f65771b15a7462a8574404f3b450af0 P a88e92ebea134aecbf7fd8de8e29dae9: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260619 02:51:11.012131 27074 raft_consensus.cc:348] T 7f65771b15a7462a8574404f3b450af0 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } } peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } }
I20260619 02:51:11.012207 27075 raft_consensus.cc:348] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } } peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } }
I20260619 02:51:11.012609 27074 raft_consensus.cc:374] T 7f65771b15a7462a8574404f3b450af0 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:11.012610 27075 raft_consensus.cc:374] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:11.012717 27075 raft_consensus.cc:729] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1d869606d534bd488f4f667e355a57a, State: Initialized, Role: FOLLOWER
I20260619 02:51:11.012722 27074 raft_consensus.cc:729] T 7f65771b15a7462a8574404f3b450af0 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a6db57534424adabd9ce75da8a7c4b3, State: Initialized, Role: FOLLOWER
I20260619 02:51:11.013378 27074 consensus_queue.cc:260] T 7f65771b15a7462a8574404f3b450af0 P 6a6db57534424adabd9ce75da8a7c4b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } } peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } }
I20260619 02:51:11.013420 27075 consensus_queue.cc:260] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } } peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } }
I20260619 02:51:11.015306 27074 ts_tablet_manager.cc:1428] T 7f65771b15a7462a8574404f3b450af0 P 6a6db57534424adabd9ce75da8a7c4b3: Time spent starting tablet: real 0.014s user 0.003s sys 0.013s
I20260619 02:51:11.015744 27075 ts_tablet_manager.cc:1428] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a: Time spent starting tablet: real 0.014s user 0.006s sys 0.008s
I20260619 02:51:11.023504 27081 raft_consensus.cc:482] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:51:11.023720 27081 raft_consensus.cc:504] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } } peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } }
I20260619 02:51:11.025171 27081 leader_election.cc:288] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a6db57534424adabd9ce75da8a7c4b3 (127.18.134.2:33961), a88e92ebea134aecbf7fd8de8e29dae9 (127.18.134.3:33887)
I20260619 02:51:11.033753 27070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:11.025864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:59275 (local address 127.18.134.3:33887)
0619 02:51:11.026036 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:11.026042 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:11.026438 (+ 396us) server_negotiation.cc:407] Connection header received
0619 02:51:11.026557 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:11.026562 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:11.026668 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:11.026748 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:11.027349 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.028386 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.031447 (+ 3061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.032650 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.032816 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:11.032885 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:11.033387 (+ 502us) server_negotiation.cc:299] Negotiation successful
0619 02:51:11.033509 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 02:51:11.033766 27069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:11.025586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:39125 (local address 127.18.134.2:33961)
0619 02:51:11.025941 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:11.025947 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:11.026256 (+ 309us) server_negotiation.cc:407] Connection header received
0619 02:51:11.026433 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:11.026437 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:11.026518 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:11.026693 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:11.027300 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.028386 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.031652 (+ 3266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.032842 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.032976 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:11.033023 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:11.033449 (+ 426us) server_negotiation.cc:299] Negotiation successful
0619 02:51:11.033567 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227}
I20260619 02:51:11.033854 27083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:11.025554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:33887 (local address 127.18.134.1:59275)
0619 02:51:11.026352 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:11.026366 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:11.026459 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:11.026753 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:11.026757 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:11.026775 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:11.027213 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:11.027223 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.028525 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.028529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:11.031243 (+ 2714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:11.031258 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.032871 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.032878 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:11.033147 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:11.033188 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 02:51:11.033296 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 02:51:11.033425 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"mutex_wait_us":143,"thread_start_us":546,"threads_started":1}
I20260619 02:51:11.033989 27082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:11.025387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:33961 (local address 127.18.134.1:39125)
0619 02:51:11.026126 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:11.026159 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:11.026310 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:11.026706 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:11.026711 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:11.026737 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:11.027156 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:11.027172 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.028524 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.028529 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:11.031243 (+ 2714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:11.031258 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.033149 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.033156 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:11.033308 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:11.033335 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 02:51:11.033411 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 02:51:11.033508 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"mutex_wait_us":162,"thread_start_us":420,"threads_started":1}
I20260619 02:51:11.034937 26863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f65771b15a7462a8574404f3b450af0" candidate_uuid: "a1d869606d534bd488f4f667e355a57a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a6db57534424adabd9ce75da8a7c4b3" is_pre_election: true
I20260619 02:51:11.034989 27003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f65771b15a7462a8574404f3b450af0" candidate_uuid: "a1d869606d534bd488f4f667e355a57a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" is_pre_election: true
I20260619 02:51:11.035391 26863 raft_consensus.cc:2454] T 7f65771b15a7462a8574404f3b450af0 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1d869606d534bd488f4f667e355a57a in term 0.
I20260619 02:51:11.035418 27003 raft_consensus.cc:2454] T 7f65771b15a7462a8574404f3b450af0 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1d869606d534bd488f4f667e355a57a in term 0.
I20260619 02:51:11.035974 26653 leader_election.cc:302] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a6db57534424adabd9ce75da8a7c4b3, a1d869606d534bd488f4f667e355a57a; no voters:
I20260619 02:51:11.036358 27081 raft_consensus.cc:2781] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:51:11.036502 27081 raft_consensus.cc:482] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:51:11.036597 27081 raft_consensus.cc:3037] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:11.038688 27081 raft_consensus.cc:504] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } } peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } }
I20260619 02:51:11.039304 27081 leader_election.cc:288] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a [CANDIDATE]: Term 1 election: Requested vote from peers 6a6db57534424adabd9ce75da8a7c4b3 (127.18.134.2:33961), a88e92ebea134aecbf7fd8de8e29dae9 (127.18.134.3:33887)
I20260619 02:51:11.039692 26863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f65771b15a7462a8574404f3b450af0" candidate_uuid: "a1d869606d534bd488f4f667e355a57a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a6db57534424adabd9ce75da8a7c4b3"
I20260619 02:51:11.039840 27003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f65771b15a7462a8574404f3b450af0" candidate_uuid: "a1d869606d534bd488f4f667e355a57a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a88e92ebea134aecbf7fd8de8e29dae9"
I20260619 02:51:11.039904 26863 raft_consensus.cc:3037] T 7f65771b15a7462a8574404f3b450af0 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:11.040033 27003 raft_consensus.cc:3037] T 7f65771b15a7462a8574404f3b450af0 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:11.043056 26863 raft_consensus.cc:2454] T 7f65771b15a7462a8574404f3b450af0 P 6a6db57534424adabd9ce75da8a7c4b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1d869606d534bd488f4f667e355a57a in term 1.
I20260619 02:51:11.043056 27003 raft_consensus.cc:2454] T 7f65771b15a7462a8574404f3b450af0 P a88e92ebea134aecbf7fd8de8e29dae9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1d869606d534bd488f4f667e355a57a in term 1.
I20260619 02:51:11.043500 26653 leader_election.cc:302] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6a6db57534424adabd9ce75da8a7c4b3, a1d869606d534bd488f4f667e355a57a, a88e92ebea134aecbf7fd8de8e29dae9; no voters:
I20260619 02:51:11.043820 27081 raft_consensus.cc:2781] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:51:11.044278 27081 raft_consensus.cc:686] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a [term 1 LEADER]: Becoming Leader. State: Replica: a1d869606d534bd488f4f667e355a57a, State: Running, Role: LEADER
I20260619 02:51:11.044821 27081 consensus_queue.cc:237] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } } peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } }
I20260619 02:51:11.049567 26561 catalog_manager.cc:5495] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a reported cstate change: term changed from 0 to 1, leader changed from <none> to a1d869606d534bd488f4f667e355a57a (127.18.134.1). New cstate: current_term: 1 leader_uuid: "a1d869606d534bd488f4f667e355a57a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } health_report { overall_health: HEALTHY } } }
I20260619 02:51:11.079888 27072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:11.075123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34032 (local address 127.18.134.1:33759)
0619 02:51:11.075283 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:11.075288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:11.075346 (+ 58us) server_negotiation.cc:407] Connection header received
0619 02:51:11.075529 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:11.075533 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:11.075618 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:11.075738 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:51:11.076017 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.076668 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.077962 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.078366 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.078457 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:11.078903 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:51:11.079430 (+ 527us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:51:11.079544 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:11.079629 (+ 85us) server_negotiation.cc:299] Negotiation successful
0619 02:51:11.079692 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 02:51:11.087911 27003 raft_consensus.cc:1261] T 7f65771b15a7462a8574404f3b450af0 P a88e92ebea134aecbf7fd8de8e29dae9 [term 1 FOLLOWER]: Refusing update from remote peer a1d869606d534bd488f4f667e355a57a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:51:11.087906 26863 raft_consensus.cc:1261] T 7f65771b15a7462a8574404f3b450af0 P 6a6db57534424adabd9ce75da8a7c4b3 [term 1 FOLLOWER]: Refusing update from remote peer a1d869606d534bd488f4f667e355a57a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:51:11.088822 27081 consensus_queue.cc:1035] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:51:11.089088 27084 consensus_queue.cc:1035] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:51:11.113891 26561 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:51:11.114122 26561 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:51:11.117389 26561 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de7f10a123774146b91cb4077e5aebb1: Sending DeleteTablet for 3 replicas of tablet 7f65771b15a7462a8574404f3b450af0
I20260619 02:51:11.118451 26839 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f65771b15a7462a8574404f3b450af0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:51:11 UTC) from {username='oryx', principal='oryx/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:48246
I20260619 02:51:11.118592 26700 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f65771b15a7462a8574404f3b450af0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:51:11 UTC) from {username='oryx', principal='oryx/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:34020
I20260619 02:51:11.118584 26978 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f65771b15a7462a8574404f3b450af0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:51:11 UTC) from {username='oryx', principal='oryx/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:57604
I20260619 02:51:11.121222 26561 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:51:11.122974 26561 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:51:11.129837 27097 tablet_replica.cc:330] T 7f65771b15a7462a8574404f3b450af0 P 6a6db57534424adabd9ce75da8a7c4b3: stopping tablet replica
I20260619 02:51:11.130218 27099 tablet_replica.cc:330] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a: stopping tablet replica
I20260619 02:51:11.130476 27097 raft_consensus.cc:2229] T 7f65771b15a7462a8574404f3b450af0 P 6a6db57534424adabd9ce75da8a7c4b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:51:11.130820 27099 raft_consensus.cc:2229] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:51:11.131032 27097 raft_consensus.cc:2258] T 7f65771b15a7462a8574404f3b450af0 P 6a6db57534424adabd9ce75da8a7c4b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:51:11.131747 27098 tablet_replica.cc:330] T 7f65771b15a7462a8574404f3b450af0 P a88e92ebea134aecbf7fd8de8e29dae9: stopping tablet replica
I20260619 02:51:11.132134 27098 raft_consensus.cc:2229] T 7f65771b15a7462a8574404f3b450af0 P a88e92ebea134aecbf7fd8de8e29dae9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:51:11.132550 27098 raft_consensus.cc:2258] T 7f65771b15a7462a8574404f3b450af0 P a88e92ebea134aecbf7fd8de8e29dae9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:51:11.132565 26700 tablet_service.cc:1467] Processing CreateTablet for tablet ffee90817f2c491ba061912298ede5e0 (DEFAULT_TABLE table=test-table [id=e40b9473dbbc41b5ae9844346e214872]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:51:11.133016 26700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffee90817f2c491ba061912298ede5e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:11.134419 27097 ts_tablet_manager.cc:1905] T 7f65771b15a7462a8574404f3b450af0 P 6a6db57534424adabd9ce75da8a7c4b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:51:11.134419 27098 ts_tablet_manager.cc:1905] T 7f65771b15a7462a8574404f3b450af0 P a88e92ebea134aecbf7fd8de8e29dae9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:51:11.141247 27097 ts_tablet_manager.cc:1918] T 7f65771b15a7462a8574404f3b450af0 P 6a6db57534424adabd9ce75da8a7c4b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:51:11.141247 27098 ts_tablet_manager.cc:1918] T 7f65771b15a7462a8574404f3b450af0 P a88e92ebea134aecbf7fd8de8e29dae9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:51:11.141422 27098 log.cc:1196] T 7f65771b15a7462a8574404f3b450af0 P a88e92ebea134aecbf7fd8de8e29dae9: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-2/wal/wals/7f65771b15a7462a8574404f3b450af0
I20260619 02:51:11.141536 27097 log.cc:1196] T 7f65771b15a7462a8574404f3b450af0 P 6a6db57534424adabd9ce75da8a7c4b3: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-1/wal/wals/7f65771b15a7462a8574404f3b450af0
I20260619 02:51:11.142081 27098 ts_tablet_manager.cc:1939] T 7f65771b15a7462a8574404f3b450af0 P a88e92ebea134aecbf7fd8de8e29dae9: Deleting consensus metadata
I20260619 02:51:11.142148 27097 ts_tablet_manager.cc:1939] T 7f65771b15a7462a8574404f3b450af0 P 6a6db57534424adabd9ce75da8a7c4b3: Deleting consensus metadata
I20260619 02:51:11.142141 26839 tablet_service.cc:1467] Processing CreateTablet for tablet ffee90817f2c491ba061912298ede5e0 (DEFAULT_TABLE table=test-table [id=e40b9473dbbc41b5ae9844346e214872]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:51:11.142700 26839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffee90817f2c491ba061912298ede5e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:11.144126 26550 catalog_manager.cc:4841] TS 6a6db57534424adabd9ce75da8a7c4b3 (127.18.134.2:33961): tablet 7f65771b15a7462a8574404f3b450af0 (table test-table [id=0aa87dcec312428d879b6e903133919b]) successfully deleted
I20260619 02:51:11.144412 26547 catalog_manager.cc:4841] TS a88e92ebea134aecbf7fd8de8e29dae9 (127.18.134.3:33887): tablet 7f65771b15a7462a8574404f3b450af0 (table test-table [id=0aa87dcec312428d879b6e903133919b]) successfully deleted
I20260619 02:51:11.147039 27075 tablet_bootstrap.cc:492] T ffee90817f2c491ba061912298ede5e0 P a1d869606d534bd488f4f667e355a57a: Bootstrap starting.
I20260619 02:51:11.147151 26978 tablet_service.cc:1467] Processing CreateTablet for tablet ffee90817f2c491ba061912298ede5e0 (DEFAULT_TABLE table=test-table [id=e40b9473dbbc41b5ae9844346e214872]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:51:11.147644 26978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffee90817f2c491ba061912298ede5e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:11.148778 27075 tablet_bootstrap.cc:654] T ffee90817f2c491ba061912298ede5e0 P a1d869606d534bd488f4f667e355a57a: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:11.152946 27099 raft_consensus.cc:2258] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:51:11.153533 27074 tablet_bootstrap.cc:492] T ffee90817f2c491ba061912298ede5e0 P 6a6db57534424adabd9ce75da8a7c4b3: Bootstrap starting.
I20260619 02:51:11.154906 27073 tablet_bootstrap.cc:492] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9: Bootstrap starting.
I20260619 02:51:11.155196 27074 tablet_bootstrap.cc:654] T ffee90817f2c491ba061912298ede5e0 P 6a6db57534424adabd9ce75da8a7c4b3: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:11.156386 27073 tablet_bootstrap.cc:654] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:11.160336 27099 ts_tablet_manager.cc:1905] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:51:11.167654 27075 tablet_bootstrap.cc:492] T ffee90817f2c491ba061912298ede5e0 P a1d869606d534bd488f4f667e355a57a: No bootstrap required, opened a new log
I20260619 02:51:11.167937 27075 ts_tablet_manager.cc:1397] T ffee90817f2c491ba061912298ede5e0 P a1d869606d534bd488f4f667e355a57a: Time spent bootstrapping tablet: real 0.021s user 0.009s sys 0.007s
I20260619 02:51:11.168372 27073 tablet_bootstrap.cc:492] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9: No bootstrap required, opened a new log
I20260619 02:51:11.168632 27073 ts_tablet_manager.cc:1397] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.012s
I20260619 02:51:11.169487 27073 raft_consensus.cc:348] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:11.169878 27073 raft_consensus.cc:374] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:11.170022 27073 raft_consensus.cc:729] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a88e92ebea134aecbf7fd8de8e29dae9, State: Initialized, Role: FOLLOWER
I20260619 02:51:11.169987 27074 tablet_bootstrap.cc:492] T ffee90817f2c491ba061912298ede5e0 P 6a6db57534424adabd9ce75da8a7c4b3: No bootstrap required, opened a new log
I20260619 02:51:11.170285 27074 ts_tablet_manager.cc:1397] T ffee90817f2c491ba061912298ede5e0 P 6a6db57534424adabd9ce75da8a7c4b3: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.006s
I20260619 02:51:11.170177 27075 raft_consensus.cc:348] T ffee90817f2c491ba061912298ede5e0 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:11.170388 27075 raft_consensus.cc:374] T ffee90817f2c491ba061912298ede5e0 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:11.170537 27075 raft_consensus.cc:729] T ffee90817f2c491ba061912298ede5e0 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1d869606d534bd488f4f667e355a57a, State: Initialized, Role: FOLLOWER
I20260619 02:51:11.170531 27073 consensus_queue.cc:260] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:11.170934 27075 consensus_queue.cc:260] T ffee90817f2c491ba061912298ede5e0 P a1d869606d534bd488f4f667e355a57a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:11.171166 27074 raft_consensus.cc:348] T ffee90817f2c491ba061912298ede5e0 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:11.171433 27074 raft_consensus.cc:374] T ffee90817f2c491ba061912298ede5e0 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:11.171579 27074 raft_consensus.cc:729] T ffee90817f2c491ba061912298ede5e0 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a6db57534424adabd9ce75da8a7c4b3, State: Initialized, Role: FOLLOWER
I20260619 02:51:11.171582 27075 ts_tablet_manager.cc:1428] T ffee90817f2c491ba061912298ede5e0 P a1d869606d534bd488f4f667e355a57a: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260619 02:51:11.172542 27073 ts_tablet_manager.cc:1428] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260619 02:51:11.172183 27074 consensus_queue.cc:260] T ffee90817f2c491ba061912298ede5e0 P 6a6db57534424adabd9ce75da8a7c4b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:11.173370 27074 ts_tablet_manager.cc:1428] T ffee90817f2c491ba061912298ede5e0 P 6a6db57534424adabd9ce75da8a7c4b3: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 02:51:11.176079 27099 ts_tablet_manager.cc:1918] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:51:11.176260 27099 log.cc:1196] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-0/wal/wals/7f65771b15a7462a8574404f3b450af0
I20260619 02:51:11.176807 27099 ts_tablet_manager.cc:1939] T 7f65771b15a7462a8574404f3b450af0 P a1d869606d534bd488f4f667e355a57a: Deleting consensus metadata
I20260619 02:51:11.178385 26547 catalog_manager.cc:4841] TS a1d869606d534bd488f4f667e355a57a (127.18.134.1:33759): tablet 7f65771b15a7462a8574404f3b450af0 (table test-table [id=0aa87dcec312428d879b6e903133919b]) successfully deleted
I20260619 02:51:11.256089 27079 raft_consensus.cc:482] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:51:11.256445 27079 raft_consensus.cc:504] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:11.258327 27079 leader_election.cc:288] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1d869606d534bd488f4f667e355a57a (127.18.134.1:33759), 6a6db57534424adabd9ce75da8a7c4b3 (127.18.134.2:33961)
I20260619 02:51:11.266271 27069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:11.258880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:43701 (local address 127.18.134.2:33961)
0619 02:51:11.259063 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:11.259069 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:11.260104 (+ 1035us) server_negotiation.cc:407] Connection header received
0619 02:51:11.260258 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:11.260263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:11.260335 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:11.260415 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:11.261108 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.262155 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.264258 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.265226 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.265381 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:11.265453 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:11.265831 (+ 378us) server_negotiation.cc:299] Negotiation successful
0619 02:51:11.265968 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 02:51:11.266884 27072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:11.258980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:35461 (local address 127.18.134.1:33759)
0619 02:51:11.259162 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:11.259168 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:11.259740 (+ 572us) server_negotiation.cc:407] Connection header received
0619 02:51:11.260082 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:11.260088 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:11.260178 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:11.260293 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:11.260973 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.262176 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.264791 (+ 2615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.265904 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.266043 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:11.266108 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:11.266483 (+ 375us) server_negotiation.cc:299] Negotiation successful
0619 02:51:11.266616 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 02:51:11.267073 27104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:11.259353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:33961 (local address 127.18.134.3:43701)
0619 02:51:11.260008 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:11.260023 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:11.260139 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:11.260556 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:11.260561 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:11.260581 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:11.260939 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:11.260949 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.262321 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.262326 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:11.264100 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:11.264113 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.265434 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.265441 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:11.265611 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:11.265646 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 02:51:11.266504 (+ 858us) client_negotiation.cc:240] Negotiation successful
0619 02:51:11.266673 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":103,"thread_start_us":391,"threads_started":1}
I20260619 02:51:11.267560 27103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:11.258609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:33759 (local address 127.18.134.3:35461)
0619 02:51:11.259623 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:11.259640 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:11.259863 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:11.260319 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:11.260323 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:11.260346 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:11.260772 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:11.260783 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.262307 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.262313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:11.264627 (+ 2314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:11.264642 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.266188 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.266195 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:11.266316 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:11.266342 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 02:51:11.267056 (+ 714us) client_negotiation.cc:240] Negotiation successful
0619 02:51:11.267204 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":897,"mutex_wait_us":53,"thread_start_us":700,"threads_started":1}
I20260619 02:51:11.267839 26863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffee90817f2c491ba061912298ede5e0" candidate_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a6db57534424adabd9ce75da8a7c4b3" is_pre_election: true
I20260619 02:51:11.268123 26863 raft_consensus.cc:2454] T ffee90817f2c491ba061912298ede5e0 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a88e92ebea134aecbf7fd8de8e29dae9 in term 0.
I20260619 02:51:11.268599 26932 leader_election.cc:302] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a6db57534424adabd9ce75da8a7c4b3, a88e92ebea134aecbf7fd8de8e29dae9; no voters:
I20260619 02:51:11.268488 26720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffee90817f2c491ba061912298ede5e0" candidate_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1d869606d534bd488f4f667e355a57a" is_pre_election: true
I20260619 02:51:11.268953 27079 raft_consensus.cc:2781] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:51:11.268931 26720 raft_consensus.cc:2454] T ffee90817f2c491ba061912298ede5e0 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a88e92ebea134aecbf7fd8de8e29dae9 in term 0.
I20260619 02:51:11.269058 27079 raft_consensus.cc:482] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:51:11.269160 27079 raft_consensus.cc:3037] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:11.271334 27079 raft_consensus.cc:504] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:11.271941 27079 leader_election.cc:288] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9 [CANDIDATE]: Term 1 election: Requested vote from peers a1d869606d534bd488f4f667e355a57a (127.18.134.1:33759), 6a6db57534424adabd9ce75da8a7c4b3 (127.18.134.2:33961)
I20260619 02:51:11.272507 26720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffee90817f2c491ba061912298ede5e0" candidate_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1d869606d534bd488f4f667e355a57a"
I20260619 02:51:11.272712 26720 raft_consensus.cc:3037] T ffee90817f2c491ba061912298ede5e0 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:11.272902 26863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffee90817f2c491ba061912298ede5e0" candidate_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a6db57534424adabd9ce75da8a7c4b3"
I20260619 02:51:11.273115 26863 raft_consensus.cc:3037] T ffee90817f2c491ba061912298ede5e0 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:11.275900 26863 raft_consensus.cc:2454] T ffee90817f2c491ba061912298ede5e0 P 6a6db57534424adabd9ce75da8a7c4b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a88e92ebea134aecbf7fd8de8e29dae9 in term 1.
I20260619 02:51:11.275918 26720 raft_consensus.cc:2454] T ffee90817f2c491ba061912298ede5e0 P a1d869606d534bd488f4f667e355a57a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a88e92ebea134aecbf7fd8de8e29dae9 in term 1.
I20260619 02:51:11.276374 26929 leader_election.cc:302] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6a6db57534424adabd9ce75da8a7c4b3, a1d869606d534bd488f4f667e355a57a, a88e92ebea134aecbf7fd8de8e29dae9; no voters:
I20260619 02:51:11.276727 27079 raft_consensus.cc:2781] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:51:11.277184 27079 raft_consensus.cc:686] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9 [term 1 LEADER]: Becoming Leader. State: Replica: a88e92ebea134aecbf7fd8de8e29dae9, State: Running, Role: LEADER
I20260619 02:51:11.277601 27079 consensus_queue.cc:237] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:11.281464 26563 catalog_manager.cc:5495] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9 reported cstate change: term changed from 0 to 1, leader changed from <none> to a88e92ebea134aecbf7fd8de8e29dae9 (127.18.134.3). New cstate: current_term: 1 leader_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } health_report { overall_health: HEALTHY } } }
Jun 19 02:51:11 dist-test-slave-9mr0 krb5kdc[26518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837468, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.62@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM
I20260619 02:51:11.336481 27112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:11.321041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:37135 (local address 127.0.0.1:36728)
0619 02:51:11.321737 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:11.321755 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:11.321974 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:11.322307 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:11.322312 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:11.323242 (+ 930us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:11.323558 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:11.323570 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.324538 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.324544 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:11.326048 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:11.326062 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.326888 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.326895 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:11.327063 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:11.327805 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:11.327827 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:11.330478 (+ 2651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:11.333640 (+ 3162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:11.333654 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:11.333662 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:11.334215 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:11.334570 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:11.334576 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:11.334580 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:11.334788 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:11.335420 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:11.335427 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:11.335599 (+ 172us) client_negotiation.cc:769] Sending connection context
0619 02:51:11.335768 (+ 169us) client_negotiation.cc:240] Negotiation successful
0619 02:51:11.336042 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":379,"threads_started":1}
I20260619 02:51:11.336504 26657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:11.321106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36728 (local address 127.18.134.62:37135)
0619 02:51:11.321394 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:11.321401 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:11.321899 (+ 498us) server_negotiation.cc:407] Connection header received
0619 02:51:11.322107 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:11.322113 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:11.322173 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:11.322255 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:11.323693 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.324412 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.326216 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.326681 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.326777 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:11.330695 (+ 3918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:11.330701 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:11.330705 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:11.330735 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:11.333505 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:11.334333 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:11.334338 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:11.334340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:11.334462 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:11.334893 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:11.334898 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:11.334901 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:11.335228 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:11.335339 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:11.335883 (+ 544us) server_negotiation.cc:299] Negotiation successful
0619 02:51:11.336161 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":97,"server-negotiator.queue_time_us":129}
I20260619 02:51:11.341833 26563 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:36728:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 02:51:11.342993 26563 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:51:11.350077 26700 tablet_service.cc:1467] Processing CreateTablet for tablet 5e1104c0aa49444f90fe00a2a4c3c875 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7a13022b971d42edbd310a76ad479458]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:51:11.350353 26978 tablet_service.cc:1467] Processing CreateTablet for tablet 5e1104c0aa49444f90fe00a2a4c3c875 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7a13022b971d42edbd310a76ad479458]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:51:11.350505 26839 tablet_service.cc:1467] Processing CreateTablet for tablet 5e1104c0aa49444f90fe00a2a4c3c875 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7a13022b971d42edbd310a76ad479458]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 02:51:11.350723 26700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e1104c0aa49444f90fe00a2a4c3c875. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:11.350900 26978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e1104c0aa49444f90fe00a2a4c3c875. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:11.350993 26839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e1104c0aa49444f90fe00a2a4c3c875. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:11.357796 27075 tablet_bootstrap.cc:492] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a: Bootstrap starting.
I20260619 02:51:11.358325 27074 tablet_bootstrap.cc:492] T 5e1104c0aa49444f90fe00a2a4c3c875 P 6a6db57534424adabd9ce75da8a7c4b3: Bootstrap starting.
I20260619 02:51:11.359614 27075 tablet_bootstrap.cc:654] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:11.359761 27074 tablet_bootstrap.cc:654] T 5e1104c0aa49444f90fe00a2a4c3c875 P 6a6db57534424adabd9ce75da8a7c4b3: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:11.360751 27073 tablet_bootstrap.cc:492] T 5e1104c0aa49444f90fe00a2a4c3c875 P a88e92ebea134aecbf7fd8de8e29dae9: Bootstrap starting.
I20260619 02:51:11.361944 27073 tablet_bootstrap.cc:654] T 5e1104c0aa49444f90fe00a2a4c3c875 P a88e92ebea134aecbf7fd8de8e29dae9: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:11.364410 27073 tablet_bootstrap.cc:492] T 5e1104c0aa49444f90fe00a2a4c3c875 P a88e92ebea134aecbf7fd8de8e29dae9: No bootstrap required, opened a new log
I20260619 02:51:11.364573 27073 ts_tablet_manager.cc:1397] T 5e1104c0aa49444f90fe00a2a4c3c875 P a88e92ebea134aecbf7fd8de8e29dae9: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260619 02:51:11.365337 27073 raft_consensus.cc:348] T 5e1104c0aa49444f90fe00a2a4c3c875 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:11.365579 27073 raft_consensus.cc:374] T 5e1104c0aa49444f90fe00a2a4c3c875 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:11.365643 27073 raft_consensus.cc:729] T 5e1104c0aa49444f90fe00a2a4c3c875 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a88e92ebea134aecbf7fd8de8e29dae9, State: Initialized, Role: FOLLOWER
I20260619 02:51:11.366084 27073 consensus_queue.cc:260] T 5e1104c0aa49444f90fe00a2a4c3c875 P a88e92ebea134aecbf7fd8de8e29dae9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:11.366758 27073 ts_tablet_manager.cc:1428] T 5e1104c0aa49444f90fe00a2a4c3c875 P a88e92ebea134aecbf7fd8de8e29dae9: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:51:11.366600 27079 tablet_replica.cc:439] T 5e1104c0aa49444f90fe00a2a4c3c875 P a88e92ebea134aecbf7fd8de8e29dae9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } } }
I20260619 02:51:11.366962 27079 tablet_replica.cc:442] T 5e1104c0aa49444f90fe00a2a4c3c875 P a88e92ebea134aecbf7fd8de8e29dae9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:51:11.371770 27075 tablet_bootstrap.cc:492] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a: No bootstrap required, opened a new log
I20260619 02:51:11.371963 27075 ts_tablet_manager.cc:1397] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.006s
I20260619 02:51:11.372742 27075 raft_consensus.cc:348] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:11.372993 27075 raft_consensus.cc:374] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:11.373078 27075 raft_consensus.cc:729] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1d869606d534bd488f4f667e355a57a, State: Initialized, Role: FOLLOWER
I20260619 02:51:11.373317 27075 consensus_queue.cc:260] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:11.373915 27075 ts_tablet_manager.cc:1428] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260619 02:51:11.373749 27092 tablet_replica.cc:439] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } } }
I20260619 02:51:11.374120 27092 tablet_replica.cc:442] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a: This TxnStatusTablet replica's current role is: FOLLOWER
W20260619 02:51:11.374836 26771 tablet.cc:2369] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:51:11.376266 26910 tablet.cc:2369] T ffee90817f2c491ba061912298ede5e0 P 6a6db57534424adabd9ce75da8a7c4b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:51:11.377846 27074 tablet_bootstrap.cc:492] T 5e1104c0aa49444f90fe00a2a4c3c875 P 6a6db57534424adabd9ce75da8a7c4b3: No bootstrap required, opened a new log
I20260619 02:51:11.378024 27074 ts_tablet_manager.cc:1397] T 5e1104c0aa49444f90fe00a2a4c3c875 P 6a6db57534424adabd9ce75da8a7c4b3: Time spent bootstrapping tablet: real 0.020s user 0.010s sys 0.000s
I20260619 02:51:11.378765 27074 raft_consensus.cc:348] T 5e1104c0aa49444f90fe00a2a4c3c875 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:11.379001 27074 raft_consensus.cc:374] T 5e1104c0aa49444f90fe00a2a4c3c875 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:11.379074 27074 raft_consensus.cc:729] T 5e1104c0aa49444f90fe00a2a4c3c875 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a6db57534424adabd9ce75da8a7c4b3, State: Initialized, Role: FOLLOWER
I20260619 02:51:11.379307 27074 consensus_queue.cc:260] T 5e1104c0aa49444f90fe00a2a4c3c875 P 6a6db57534424adabd9ce75da8a7c4b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:11.379827 27074 ts_tablet_manager.cc:1428] T 5e1104c0aa49444f90fe00a2a4c3c875 P 6a6db57534424adabd9ce75da8a7c4b3: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260619 02:51:11.379739 27080 tablet_replica.cc:439] T 5e1104c0aa49444f90fe00a2a4c3c875 P 6a6db57534424adabd9ce75da8a7c4b3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } } }
I20260619 02:51:11.380134 27080 tablet_replica.cc:442] T 5e1104c0aa49444f90fe00a2a4c3c875 P 6a6db57534424adabd9ce75da8a7c4b3: This TxnStatusTablet replica's current role is: FOLLOWER
W20260619 02:51:11.402726 27049 tablet.cc:2369] T 5e1104c0aa49444f90fe00a2a4c3c875 P a88e92ebea134aecbf7fd8de8e29dae9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:51:11.423259 27092 raft_consensus.cc:482] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:51:11.423467 27092 raft_consensus.cc:504] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:11.424166 27092 leader_election.cc:288] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a6db57534424adabd9ce75da8a7c4b3 (127.18.134.2:33961), a88e92ebea134aecbf7fd8de8e29dae9 (127.18.134.3:33887)
I20260619 02:51:11.424719 27003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e1104c0aa49444f90fe00a2a4c3c875" candidate_uuid: "a1d869606d534bd488f4f667e355a57a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" is_pre_election: true
I20260619 02:51:11.424732 26863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e1104c0aa49444f90fe00a2a4c3c875" candidate_uuid: "a1d869606d534bd488f4f667e355a57a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a6db57534424adabd9ce75da8a7c4b3" is_pre_election: true
I20260619 02:51:11.424935 27003 raft_consensus.cc:2454] T 5e1104c0aa49444f90fe00a2a4c3c875 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1d869606d534bd488f4f667e355a57a in term 0.
I20260619 02:51:11.425004 26863 raft_consensus.cc:2454] T 5e1104c0aa49444f90fe00a2a4c3c875 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1d869606d534bd488f4f667e355a57a in term 0.
I20260619 02:51:11.425343 26650 leader_election.cc:302] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a1d869606d534bd488f4f667e355a57a, a88e92ebea134aecbf7fd8de8e29dae9; no voters:
I20260619 02:51:11.425601 27092 raft_consensus.cc:2781] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:51:11.425727 27092 raft_consensus.cc:482] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:51:11.425827 27092 raft_consensus.cc:3037] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:11.428197 27092 raft_consensus.cc:504] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:11.428706 27092 leader_election.cc:288] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a [CANDIDATE]: Term 1 election: Requested vote from peers 6a6db57534424adabd9ce75da8a7c4b3 (127.18.134.2:33961), a88e92ebea134aecbf7fd8de8e29dae9 (127.18.134.3:33887)
I20260619 02:51:11.429178 27003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e1104c0aa49444f90fe00a2a4c3c875" candidate_uuid: "a1d869606d534bd488f4f667e355a57a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a88e92ebea134aecbf7fd8de8e29dae9"
I20260619 02:51:11.429160 26863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e1104c0aa49444f90fe00a2a4c3c875" candidate_uuid: "a1d869606d534bd488f4f667e355a57a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a6db57534424adabd9ce75da8a7c4b3"
I20260619 02:51:11.429363 27003 raft_consensus.cc:3037] T 5e1104c0aa49444f90fe00a2a4c3c875 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:11.429379 26863 raft_consensus.cc:3037] T 5e1104c0aa49444f90fe00a2a4c3c875 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:11.432205 26863 raft_consensus.cc:2454] T 5e1104c0aa49444f90fe00a2a4c3c875 P 6a6db57534424adabd9ce75da8a7c4b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1d869606d534bd488f4f667e355a57a in term 1.
I20260619 02:51:11.432219 27003 raft_consensus.cc:2454] T 5e1104c0aa49444f90fe00a2a4c3c875 P a88e92ebea134aecbf7fd8de8e29dae9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1d869606d534bd488f4f667e355a57a in term 1.
I20260619 02:51:11.432639 26653 leader_election.cc:302] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a6db57534424adabd9ce75da8a7c4b3, a1d869606d534bd488f4f667e355a57a; no voters:
I20260619 02:51:11.432966 27092 raft_consensus.cc:2781] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:51:11.433130 27092 raft_consensus.cc:686] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a [term 1 LEADER]: Becoming Leader. State: Replica: a1d869606d534bd488f4f667e355a57a, State: Running, Role: LEADER
I20260619 02:51:11.433382 27092 consensus_queue.cc:237] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:11.434939 27084 tablet_replica.cc:439] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a: TxnStatusTablet state changed. Reason: New leader a1d869606d534bd488f4f667e355a57a. Latest consensus state: current_term: 1 leader_uuid: "a1d869606d534bd488f4f667e355a57a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } } }
I20260619 02:51:11.435292 27084 tablet_replica.cc:442] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:51:11.436209 27117 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 02:51:11.436460 27117 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 02:51:11.436461 26563 catalog_manager.cc:5495] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a reported cstate change: term changed from 0 to 1, leader changed from <none> to a1d869606d534bd488f4f667e355a57a (127.18.134.1). New cstate: current_term: 1 leader_uuid: "a1d869606d534bd488f4f667e355a57a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } health_report { overall_health: UNKNOWN } } }
I20260619 02:51:11.455652 27072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:11.449790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34034 (local address 127.18.134.1:33759)
0619 02:51:11.449944 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:11.449948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:11.450297 (+ 349us) server_negotiation.cc:407] Connection header received
0619 02:51:11.450536 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:11.450540 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:11.450682 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:11.450770 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:51:11.451184 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.452000 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.453826 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.454369 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.454475 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:11.454997 (+ 522us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:51:11.455104 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:51:11.455172 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:11.455343 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 02:51:11.455406 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 02:51:11.455770 27112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:11.449809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:33759 (local address 127.0.0.1:34034)
0619 02:51:11.450205 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:11.450224 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:11.450372 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:11.450811 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:11.450816 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:11.450828 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:51:11.451039 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:11.451051 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.452134 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.452140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:11.453647 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:11.453663 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.454527 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.454533 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:11.454760 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:11.454834 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:51:11.455203 (+ 369us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:51:11.455233 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 02:51:11.455324 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 02:51:11.455470 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241}
I20260619 02:51:11.459381 27003 raft_consensus.cc:1261] T 5e1104c0aa49444f90fe00a2a4c3c875 P a88e92ebea134aecbf7fd8de8e29dae9 [term 1 FOLLOWER]: Refusing update from remote peer a1d869606d534bd488f4f667e355a57a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:51:11.459491 26863 raft_consensus.cc:1261] T 5e1104c0aa49444f90fe00a2a4c3c875 P 6a6db57534424adabd9ce75da8a7c4b3 [term 1 FOLLOWER]: Refusing update from remote peer a1d869606d534bd488f4f667e355a57a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:51:11.460035 27092 consensus_queue.cc:1035] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:51:11.460269 27084 consensus_queue.cc:1035] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:51:11.462677 27080 tablet_replica.cc:439] T 5e1104c0aa49444f90fe00a2a4c3c875 P 6a6db57534424adabd9ce75da8a7c4b3: TxnStatusTablet state changed. Reason: New leader a1d869606d534bd488f4f667e355a57a. Latest consensus state: current_term: 1 leader_uuid: "a1d869606d534bd488f4f667e355a57a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } } }
I20260619 02:51:11.462999 27080 tablet_replica.cc:442] T 5e1104c0aa49444f90fe00a2a4c3c875 P 6a6db57534424adabd9ce75da8a7c4b3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:51:11.465528 27081 tablet_replica.cc:439] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a1d869606d534bd488f4f667e355a57a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } } }
I20260619 02:51:11.465834 27081 tablet_replica.cc:442] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:51:11.466006 27087 mvcc.cc:205] Tried to move back new op lower bound from 7298406283092144128 to 7298406282993172480. Current Snapshot: MvccSnapshot[applied={T|T < 7298406283092144128 or (T in {7298406283092144128})}]
I20260619 02:51:11.467299 27089 mvcc.cc:205] Tried to move back new op lower bound from 7298406283092144128 to 7298406282993172480. Current Snapshot: MvccSnapshot[applied={T|T < 7298406283092144128}]
I20260619 02:51:11.467769 27080 tablet_replica.cc:439] T 5e1104c0aa49444f90fe00a2a4c3c875 P 6a6db57534424adabd9ce75da8a7c4b3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a1d869606d534bd488f4f667e355a57a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } } }
I20260619 02:51:11.467988 27080 tablet_replica.cc:442] T 5e1104c0aa49444f90fe00a2a4c3c875 P 6a6db57534424adabd9ce75da8a7c4b3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:51:11.475340 27079 tablet_replica.cc:439] T 5e1104c0aa49444f90fe00a2a4c3c875 P a88e92ebea134aecbf7fd8de8e29dae9: TxnStatusTablet state changed. Reason: New leader a1d869606d534bd488f4f667e355a57a. Latest consensus state: current_term: 1 leader_uuid: "a1d869606d534bd488f4f667e355a57a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } } }
I20260619 02:51:11.475654 27079 tablet_replica.cc:442] T 5e1104c0aa49444f90fe00a2a4c3c875 P a88e92ebea134aecbf7fd8de8e29dae9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:51:11.475754 27084 tablet_replica.cc:439] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a1d869606d534bd488f4f667e355a57a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } } }
I20260619 02:51:11.475983 27084 tablet_replica.cc:442] T 5e1104c0aa49444f90fe00a2a4c3c875 P a1d869606d534bd488f4f667e355a57a: This TxnStatusTablet replica's current role is: LEADER
I20260619 02:51:11.477950 27088 mvcc.cc:205] Tried to move back new op lower bound from 7298406283092144128 to 7298406282993172480. Current Snapshot: MvccSnapshot[applied={T|T < 7298406283092144128 or (T in {7298406283092144128})}]
I20260619 02:51:11.478844 27079 tablet_replica.cc:439] T 5e1104c0aa49444f90fe00a2a4c3c875 P a88e92ebea134aecbf7fd8de8e29dae9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a1d869606d534bd488f4f667e355a57a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } } }
I20260619 02:51:11.479115 27079 tablet_replica.cc:442] T 5e1104c0aa49444f90fe00a2a4c3c875 P a88e92ebea134aecbf7fd8de8e29dae9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 02:51:11.481910 27070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:11.474573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57612 (local address 127.18.134.3:33887)
0619 02:51:11.474753 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:11.474759 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:11.475062 (+ 303us) server_negotiation.cc:407] Connection header received
0619 02:51:11.475171 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:11.475176 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:11.475286 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:11.475434 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:51:11.476139 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.477326 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.479290 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.479921 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.480056 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:11.480676 (+ 620us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:51:11.481393 (+ 717us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:51:11.481500 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:11.481635 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 02:51:11.481699 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 02:51:11.499126 27072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:11.492599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:58795 (local address 127.18.134.1:33759)
0619 02:51:11.492809 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:11.492817 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:11.493326 (+ 509us) server_negotiation.cc:407] Connection header received
0619 02:51:11.493644 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:11.493648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:11.493718 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:11.493795 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:51:11.494363 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.495233 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.497225 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.497732 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.497853 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:11.498378 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:51:11.498511 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:51:11.498590 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:11.498772 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 02:51:11.498864 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 02:51:11.499385 27123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:11.492576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:33759 (local address 127.18.134.3:58795)
0619 02:51:11.493223 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:11.493243 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:11.493398 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:11.493843 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:11.493848 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:11.493861 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:51:11.494190 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:11.494203 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.495378 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.495383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:11.497086 (+ 1703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:11.497101 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.497910 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.497916 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:11.498105 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:11.498183 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:51:11.498617 (+ 434us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:51:11.498656 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 02:51:11.498745 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 02:51:11.498890 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":324,"threads_started":1}
I20260619 02:51:11.508867 26863 raft_consensus.cc:1261] T ffee90817f2c491ba061912298ede5e0 P 6a6db57534424adabd9ce75da8a7c4b3 [term 1 FOLLOWER]: Refusing update from remote peer a88e92ebea134aecbf7fd8de8e29dae9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:51:11.509176 26720 raft_consensus.cc:1261] T ffee90817f2c491ba061912298ede5e0 P a1d869606d534bd488f4f667e355a57a [term 1 FOLLOWER]: Refusing update from remote peer a88e92ebea134aecbf7fd8de8e29dae9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:51:11.509572 27105 consensus_queue.cc:1035] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:51:11.510169 27079 consensus_queue.cc:1035] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:51:11.548439 27070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:11.542169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:35767 (local address 127.18.134.3:33887)
0619 02:51:11.542321 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:11.542327 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:11.543217 (+ 890us) server_negotiation.cc:407] Connection header received
0619 02:51:11.543395 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:11.543400 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:11.543474 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:11.543543 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:51:11.544013 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.544879 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.546733 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.547215 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.547316 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:11.547760 (+ 444us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:51:11.547883 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:51:11.547950 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:11.548078 (+ 128us) server_negotiation.cc:299] Negotiation successful
0619 02:51:11.548156 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 02:51:11.548698 27127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:11.541810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:33887 (local address 127.18.134.1:35767)
0619 02:51:11.543130 (+ 1320us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:11.543147 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:11.543266 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:11.543576 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:11.543580 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:11.543592 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:51:11.543872 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:11.543888 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.544988 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.544994 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:11.546569 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:11.546584 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.547393 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.547399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:11.547517 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:11.547572 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:51:11.547969 (+ 397us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:51:11.547981 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 02:51:11.548053 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 02:51:11.548244 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1109,"thread_start_us":987,"threads_started":1}
I20260619 02:51:11.590930 26563 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:51:11.591200 26563 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:51:11.594732 26563 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de7f10a123774146b91cb4077e5aebb1: Sending DeleteTablet for 3 replicas of tablet ffee90817f2c491ba061912298ede5e0
I20260619 02:51:11.595561 26700 tablet_service.cc:1514] Processing DeleteTablet for tablet ffee90817f2c491ba061912298ede5e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:51:11 UTC) from {username='oryx', principal='oryx/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:34020
I20260619 02:51:11.595975 27099 tablet_replica.cc:330] T ffee90817f2c491ba061912298ede5e0 P a1d869606d534bd488f4f667e355a57a: stopping tablet replica
I20260619 02:51:11.595958 26839 tablet_service.cc:1514] Processing DeleteTablet for tablet ffee90817f2c491ba061912298ede5e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:51:11 UTC) from {username='oryx', principal='oryx/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:48246
I20260619 02:51:11.596200 27099 raft_consensus.cc:2229] T ffee90817f2c491ba061912298ede5e0 P a1d869606d534bd488f4f667e355a57a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:51:11.596230 27097 tablet_replica.cc:330] T ffee90817f2c491ba061912298ede5e0 P 6a6db57534424adabd9ce75da8a7c4b3: stopping tablet replica
I20260619 02:51:11.596410 27099 raft_consensus.cc:2258] T ffee90817f2c491ba061912298ede5e0 P a1d869606d534bd488f4f667e355a57a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:51:11.596419 27097 raft_consensus.cc:2229] T ffee90817f2c491ba061912298ede5e0 P 6a6db57534424adabd9ce75da8a7c4b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:51:11.596596 27097 raft_consensus.cc:2258] T ffee90817f2c491ba061912298ede5e0 P 6a6db57534424adabd9ce75da8a7c4b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:51:11.597077 26978 tablet_service.cc:1514] Processing DeleteTablet for tablet ffee90817f2c491ba061912298ede5e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:51:11 UTC) from {username='oryx', principal='oryx/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:57604
I20260619 02:51:11.597360 27098 tablet_replica.cc:330] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9: stopping tablet replica
I20260619 02:51:11.597559 27098 raft_consensus.cc:2229] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:51:11.598057 27098 raft_consensus.cc:2258] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:51:11.598057 27099 ts_tablet_manager.cc:1905] T ffee90817f2c491ba061912298ede5e0 P a1d869606d534bd488f4f667e355a57a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:51:11.599273 27098 ts_tablet_manager.cc:1905] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:51:11.602257 27097 ts_tablet_manager.cc:1905] T ffee90817f2c491ba061912298ede5e0 P 6a6db57534424adabd9ce75da8a7c4b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:51:11.612891 27099 ts_tablet_manager.cc:1918] T ffee90817f2c491ba061912298ede5e0 P a1d869606d534bd488f4f667e355a57a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:51:11.613849 27098 ts_tablet_manager.cc:1918] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:51:11.614029 27098 log.cc:1196] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-2/wal/wals/ffee90817f2c491ba061912298ede5e0
I20260619 02:51:11.614460 27098 ts_tablet_manager.cc:1939] T ffee90817f2c491ba061912298ede5e0 P a88e92ebea134aecbf7fd8de8e29dae9: Deleting consensus metadata
I20260619 02:51:11.615739 26547 catalog_manager.cc:4841] TS a88e92ebea134aecbf7fd8de8e29dae9 (127.18.134.3:33887): tablet ffee90817f2c491ba061912298ede5e0 (table test-table [id=e40b9473dbbc41b5ae9844346e214872]) successfully deleted
I20260619 02:51:11.618407 27097 ts_tablet_manager.cc:1918] T ffee90817f2c491ba061912298ede5e0 P 6a6db57534424adabd9ce75da8a7c4b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:51:11.618577 27097 log.cc:1196] T ffee90817f2c491ba061912298ede5e0 P 6a6db57534424adabd9ce75da8a7c4b3: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-1/wal/wals/ffee90817f2c491ba061912298ede5e0
I20260619 02:51:11.618997 27097 ts_tablet_manager.cc:1939] T ffee90817f2c491ba061912298ede5e0 P 6a6db57534424adabd9ce75da8a7c4b3: Deleting consensus metadata
I20260619 02:51:11.619163 27099 log.cc:1196] T ffee90817f2c491ba061912298ede5e0 P a1d869606d534bd488f4f667e355a57a: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-0/wal/wals/ffee90817f2c491ba061912298ede5e0
I20260619 02:51:11.619673 27099 ts_tablet_manager.cc:1939] T ffee90817f2c491ba061912298ede5e0 P a1d869606d534bd488f4f667e355a57a: Deleting consensus metadata
I20260619 02:51:11.620129 26550 catalog_manager.cc:4841] TS 6a6db57534424adabd9ce75da8a7c4b3 (127.18.134.2:33961): tablet ffee90817f2c491ba061912298ede5e0 (table test-table [id=e40b9473dbbc41b5ae9844346e214872]) successfully deleted
I20260619 02:51:11.620992 26547 catalog_manager.cc:4841] TS a1d869606d534bd488f4f667e355a57a (127.18.134.1:33759): tablet ffee90817f2c491ba061912298ede5e0 (table test-table [id=e40b9473dbbc41b5ae9844346e214872]) successfully deleted
Jun 19 02:51:11 dist-test-slave-9mr0 krb5kdc[26518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 02:51:11 dist-test-slave-9mr0 krb5kdc[26518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:11 dist-test-slave-9mr0 krb5kdc[26518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
W20260619 02:51:11.680823 26657 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:11.666788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36730 (local address 127.18.134.62:37135)
0619 02:51:11.666995 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:11.667002 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:11.668010 (+ 1008us) server_negotiation.cc:407] Connection header received
0619 02:51:11.668239 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:11.668246 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:11.668336 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:11.668446 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:11.670070 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.671216 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.673187 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.673803 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.673937 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:11.680549 (+ 6612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36730: BlockingRecv error: recv got EOF from 127.0.0.1:36730 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Jun 19 02:51:11 dist-test-slave-9mr0 krb5kdc[26518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM
I20260619 02:51:11.703039 26657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:11.685048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36742 (local address 127.18.134.62:37135)
0619 02:51:11.685230 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:11.685237 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:11.685260 (+ 23us) server_negotiation.cc:407] Connection header received
0619 02:51:11.685460 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:11.685466 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:11.685562 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:11.685636 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:11.687374 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.688542 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.690422 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.691122 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.691260 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:11.696187 (+ 4927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:11.696195 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:11.696201 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:11.696251 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:11.699794 (+ 3543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:11.700756 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:11.700762 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:11.700766 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:11.700929 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:11.701375 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:11.701380 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:11.701384 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:11.701777 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:11.702022 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:11.702422 (+ 400us) server_negotiation.cc:299] Negotiation successful
0619 02:51:11.702675 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 02:51:11.707088 26563 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36742:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:51:11.707796 26563 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:51:11.715610 26700 tablet_service.cc:1467] Processing CreateTablet for tablet 97db34710f924827b609603b3784b5b8 (DEFAULT_TABLE table=test-table [id=4f0123882ec844f6af946947a4b44aad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:51:11.716122 26700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97db34710f924827b609603b3784b5b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:11.716395 26978 tablet_service.cc:1467] Processing CreateTablet for tablet 97db34710f924827b609603b3784b5b8 (DEFAULT_TABLE table=test-table [id=4f0123882ec844f6af946947a4b44aad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:51:11.716805 26978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97db34710f924827b609603b3784b5b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:11.719427 26839 tablet_service.cc:1467] Processing CreateTablet for tablet 97db34710f924827b609603b3784b5b8 (DEFAULT_TABLE table=test-table [id=4f0123882ec844f6af946947a4b44aad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:51:11.719831 26839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97db34710f924827b609603b3784b5b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:11.724066 27073 tablet_bootstrap.cc:492] T 97db34710f924827b609603b3784b5b8 P a88e92ebea134aecbf7fd8de8e29dae9: Bootstrap starting.
I20260619 02:51:11.725807 27073 tablet_bootstrap.cc:654] T 97db34710f924827b609603b3784b5b8 P a88e92ebea134aecbf7fd8de8e29dae9: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:11.734465 27074 tablet_bootstrap.cc:492] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3: Bootstrap starting.
I20260619 02:51:11.734905 27075 tablet_bootstrap.cc:492] T 97db34710f924827b609603b3784b5b8 P a1d869606d534bd488f4f667e355a57a: Bootstrap starting.
I20260619 02:51:11.736099 27074 tablet_bootstrap.cc:654] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:11.736496 27075 tablet_bootstrap.cc:654] T 97db34710f924827b609603b3784b5b8 P a1d869606d534bd488f4f667e355a57a: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:11.740689 27075 tablet_bootstrap.cc:492] T 97db34710f924827b609603b3784b5b8 P a1d869606d534bd488f4f667e355a57a: No bootstrap required, opened a new log
I20260619 02:51:11.740842 27075 ts_tablet_manager.cc:1397] T 97db34710f924827b609603b3784b5b8 P a1d869606d534bd488f4f667e355a57a: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260619 02:51:11.741647 27075 raft_consensus.cc:348] T 97db34710f924827b609603b3784b5b8 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:11.741917 27075 raft_consensus.cc:374] T 97db34710f924827b609603b3784b5b8 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:11.742007 27075 raft_consensus.cc:729] T 97db34710f924827b609603b3784b5b8 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1d869606d534bd488f4f667e355a57a, State: Initialized, Role: FOLLOWER
I20260619 02:51:11.742244 27075 consensus_queue.cc:260] T 97db34710f924827b609603b3784b5b8 P a1d869606d534bd488f4f667e355a57a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:11.743280 27075 ts_tablet_manager.cc:1428] T 97db34710f924827b609603b3784b5b8 P a1d869606d534bd488f4f667e355a57a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 02:51:11.744298 27073 tablet_bootstrap.cc:492] T 97db34710f924827b609603b3784b5b8 P a88e92ebea134aecbf7fd8de8e29dae9: No bootstrap required, opened a new log
I20260619 02:51:11.744453 27073 ts_tablet_manager.cc:1397] T 97db34710f924827b609603b3784b5b8 P a88e92ebea134aecbf7fd8de8e29dae9: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.013s
I20260619 02:51:11.744966 27074 tablet_bootstrap.cc:492] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3: No bootstrap required, opened a new log
I20260619 02:51:11.745162 27074 ts_tablet_manager.cc:1397] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.002s
I20260619 02:51:11.745219 27073 raft_consensus.cc:348] T 97db34710f924827b609603b3784b5b8 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:11.745466 27073 raft_consensus.cc:374] T 97db34710f924827b609603b3784b5b8 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:11.745559 27073 raft_consensus.cc:729] T 97db34710f924827b609603b3784b5b8 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a88e92ebea134aecbf7fd8de8e29dae9, State: Initialized, Role: FOLLOWER
I20260619 02:51:11.745818 27073 consensus_queue.cc:260] T 97db34710f924827b609603b3784b5b8 P a88e92ebea134aecbf7fd8de8e29dae9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:11.746346 27073 ts_tablet_manager.cc:1428] T 97db34710f924827b609603b3784b5b8 P a88e92ebea134aecbf7fd8de8e29dae9: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260619 02:51:11.746454 27074 raft_consensus.cc:348] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:11.746821 27074 raft_consensus.cc:374] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:11.746964 27074 raft_consensus.cc:729] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a6db57534424adabd9ce75da8a7c4b3, State: Initialized, Role: FOLLOWER
I20260619 02:51:11.747287 27074 consensus_queue.cc:260] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:11.747856 27074 ts_tablet_manager.cc:1428] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260619 02:51:11.874933 27080 raft_consensus.cc:482] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:51:11.875166 27080 raft_consensus.cc:504] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:11.876900 27080 leader_election.cc:288] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1d869606d534bd488f4f667e355a57a (127.18.134.1:33759), a88e92ebea134aecbf7fd8de8e29dae9 (127.18.134.3:33887)
I20260619 02:51:11.909448 27072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:11.877786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:48169 (local address 127.18.134.1:33759)
0619 02:51:11.877975 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:11.877981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:11.896213 (+ 18232us) server_negotiation.cc:407] Connection header received
0619 02:51:11.896457 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:11.896463 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:11.896529 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:11.896582 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:11.897280 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.898344 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.900889 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.902117 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.902248 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:11.902320 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:11.909027 (+ 6707us) server_negotiation.cc:299] Negotiation successful
0619 02:51:11.909204 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 02:51:11.910218 27146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:11.877411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:33759 (local address 127.18.134.2:48169)
0619 02:51:11.896081 (+ 18670us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:11.896104 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:11.896318 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:11.896836 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:11.896843 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:11.896872 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:11.897112 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:11.897122 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.898525 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.898530 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:11.900700 (+ 2170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:11.900714 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.902500 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.902506 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:11.902711 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:11.902744 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 02:51:11.909779 (+ 7035us) client_negotiation.cc:240] Negotiation successful
0619 02:51:11.909929 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18539,"thread_start_us":18487,"threads_started":1}
I20260619 02:51:11.911109 26720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97db34710f924827b609603b3784b5b8" candidate_uuid: "6a6db57534424adabd9ce75da8a7c4b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1d869606d534bd488f4f667e355a57a" is_pre_election: true
I20260619 02:51:11.911341 27070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:11.896515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:46469 (local address 127.18.134.3:33887)
0619 02:51:11.896805 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:11.896811 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:11.903722 (+ 6911us) server_negotiation.cc:407] Connection header received
0619 02:51:11.904156 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:11.904162 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:11.904241 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:11.904300 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:11.905285 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.906384 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.908756 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.910087 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.910236 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:11.910313 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:11.910935 (+ 622us) server_negotiation.cc:299] Negotiation successful
0619 02:51:11.911071 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260619 02:51:11.911357 26720 raft_consensus.cc:2454] T 97db34710f924827b609603b3784b5b8 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a6db57534424adabd9ce75da8a7c4b3 in term 0.
I20260619 02:51:11.911891 26790 leader_election.cc:302] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a6db57534424adabd9ce75da8a7c4b3, a1d869606d534bd488f4f667e355a57a; no voters:
I20260619 02:51:11.912408 27080 raft_consensus.cc:2781] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:51:11.912515 27080 raft_consensus.cc:482] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:51:11.912601 27080 raft_consensus.cc:3037] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:11.915632 27080 raft_consensus.cc:504] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:11.916304 27080 leader_election.cc:288] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3 [CANDIDATE]: Term 1 election: Requested vote from peers a1d869606d534bd488f4f667e355a57a (127.18.134.1:33759), a88e92ebea134aecbf7fd8de8e29dae9 (127.18.134.3:33887)
I20260619 02:51:11.917229 26720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97db34710f924827b609603b3784b5b8" candidate_uuid: "6a6db57534424adabd9ce75da8a7c4b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1d869606d534bd488f4f667e355a57a"
I20260619 02:51:11.917434 26720 raft_consensus.cc:3037] T 97db34710f924827b609603b3784b5b8 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:11.920990 27147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:11.897025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:33887 (local address 127.18.134.2:46469)
0619 02:51:11.903615 (+ 6590us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:11.903637 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:11.903993 (+ 356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:11.904486 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:11.904491 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:11.904650 (+ 159us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:11.905066 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:11.905079 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.906563 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.906568 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:11.908567 (+ 1999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:11.908582 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.910528 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.910533 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:11.910755 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:11.910788 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 02:51:11.911732 (+ 944us) client_negotiation.cc:240] Negotiation successful
0619 02:51:11.911860 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6446,"thread_start_us":6698,"threads_started":1}
I20260619 02:51:11.921689 26720 raft_consensus.cc:2454] T 97db34710f924827b609603b3784b5b8 P a1d869606d534bd488f4f667e355a57a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a6db57534424adabd9ce75da8a7c4b3 in term 1.
I20260619 02:51:11.921746 27002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97db34710f924827b609603b3784b5b8" candidate_uuid: "6a6db57534424adabd9ce75da8a7c4b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" is_pre_election: true
I20260619 02:51:11.922024 27002 raft_consensus.cc:2454] T 97db34710f924827b609603b3784b5b8 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a6db57534424adabd9ce75da8a7c4b3 in term 0.
I20260619 02:51:11.922981 27002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97db34710f924827b609603b3784b5b8" candidate_uuid: "6a6db57534424adabd9ce75da8a7c4b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a88e92ebea134aecbf7fd8de8e29dae9"
I20260619 02:51:11.923178 27002 raft_consensus.cc:3037] T 97db34710f924827b609603b3784b5b8 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:11.926290 27002 raft_consensus.cc:2454] T 97db34710f924827b609603b3784b5b8 P a88e92ebea134aecbf7fd8de8e29dae9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a6db57534424adabd9ce75da8a7c4b3 in term 1.
I20260619 02:51:11.926851 26790 leader_election.cc:302] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a6db57534424adabd9ce75da8a7c4b3, a1d869606d534bd488f4f667e355a57a; no voters:
I20260619 02:51:11.927831 27080 raft_consensus.cc:2781] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:51:11.935075 27080 raft_consensus.cc:686] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3 [term 1 LEADER]: Becoming Leader. State: Replica: 6a6db57534424adabd9ce75da8a7c4b3, State: Running, Role: LEADER
I20260619 02:51:11.935577 27080 consensus_queue.cc:237] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:11.946835 26561 catalog_manager.cc:5495] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a6db57534424adabd9ce75da8a7c4b3 (127.18.134.2). New cstate: current_term: 1 leader_uuid: "6a6db57534424adabd9ce75da8a7c4b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } health_report { overall_health: UNKNOWN } } }
I20260619 02:51:11.986663 27151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:11.973601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48250 (local address 127.18.134.2:33961)
0619 02:51:11.978658 (+ 5057us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:11.978665 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:11.978692 (+ 27us) server_negotiation.cc:407] Connection header received
0619 02:51:11.978804 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:11.978809 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:11.978922 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:11.979119 (+ 197us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:51:11.980597 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.981788 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.983794 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:11.984434 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:11.984576 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:11.985093 (+ 517us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:51:11.985900 (+ 807us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:51:11.986051 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:11.986177 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 02:51:11.986260 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4869,"thread_start_us":4728,"threads_started":1}
I20260619 02:51:11.990813 27002 raft_consensus.cc:1261] T 97db34710f924827b609603b3784b5b8 P a88e92ebea134aecbf7fd8de8e29dae9 [term 1 FOLLOWER]: Refusing update from remote peer 6a6db57534424adabd9ce75da8a7c4b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:51:11.990918 26720 raft_consensus.cc:1261] T 97db34710f924827b609603b3784b5b8 P a1d869606d534bd488f4f667e355a57a [term 1 FOLLOWER]: Refusing update from remote peer 6a6db57534424adabd9ce75da8a7c4b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:51:11.991577 27148 consensus_queue.cc:1035] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:51:11.992357 27148 consensus_queue.cc:1035] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:51:12.036871 26561 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:51:12.037184 26561 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:51:12.041016 26561 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de7f10a123774146b91cb4077e5aebb1: Sending DeleteTablet for 3 replicas of tablet 97db34710f924827b609603b3784b5b8
I20260619 02:51:12.041913 26700 tablet_service.cc:1514] Processing DeleteTablet for tablet 97db34710f924827b609603b3784b5b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:51:12 UTC) from {username='oryx', principal='oryx/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:34020
I20260619 02:51:12.042238 27099 tablet_replica.cc:330] T 97db34710f924827b609603b3784b5b8 P a1d869606d534bd488f4f667e355a57a: stopping tablet replica
I20260619 02:51:12.042449 27099 raft_consensus.cc:2229] T 97db34710f924827b609603b3784b5b8 P a1d869606d534bd488f4f667e355a57a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:51:12.042733 27099 raft_consensus.cc:2258] T 97db34710f924827b609603b3784b5b8 P a1d869606d534bd488f4f667e355a57a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:51:12.042905 26839 tablet_service.cc:1514] Processing DeleteTablet for tablet 97db34710f924827b609603b3784b5b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:51:12 UTC) from {username='oryx', principal='oryx/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:48246
I20260619 02:51:12.043169 27097 tablet_replica.cc:330] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3: stopping tablet replica
I20260619 02:51:12.043319 27097 raft_consensus.cc:2229] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:51:12.043777 27097 raft_consensus.cc:2258] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:51:12.044831 26978 tablet_service.cc:1514] Processing DeleteTablet for tablet 97db34710f924827b609603b3784b5b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:51:12 UTC) from {username='oryx', principal='oryx/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:57604
I20260619 02:51:12.045135 27098 tablet_replica.cc:330] T 97db34710f924827b609603b3784b5b8 P a88e92ebea134aecbf7fd8de8e29dae9: stopping tablet replica
I20260619 02:51:12.045949 27097 ts_tablet_manager.cc:1905] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:51:12.052911 27097 ts_tablet_manager.cc:1918] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:51:12.054910 26563 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36742:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:51:12.055904 26563 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:51:12.064888 26978 tablet_service.cc:1467] Processing CreateTablet for tablet 6cd4c91eaea5493db8b164bfc9e8b7f1 (DEFAULT_TABLE table=test-table [id=4fc7b305166c4980a812941a958562cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:51:12.065431 26978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cd4c91eaea5493db8b164bfc9e8b7f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:12.067000 26700 tablet_service.cc:1467] Processing CreateTablet for tablet 6cd4c91eaea5493db8b164bfc9e8b7f1 (DEFAULT_TABLE table=test-table [id=4fc7b305166c4980a812941a958562cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:51:12.067456 26700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cd4c91eaea5493db8b164bfc9e8b7f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:12.085088 27075 tablet_bootstrap.cc:492] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a: Bootstrap starting.
I20260619 02:51:12.086944 27075 tablet_bootstrap.cc:654] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:12.093681 27073 tablet_bootstrap.cc:492] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a88e92ebea134aecbf7fd8de8e29dae9: Bootstrap starting.
I20260619 02:51:12.072352 26839 tablet_service.cc:1467] Processing CreateTablet for tablet 6cd4c91eaea5493db8b164bfc9e8b7f1 (DEFAULT_TABLE table=test-table [id=4fc7b305166c4980a812941a958562cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:51:12.094944 26839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cd4c91eaea5493db8b164bfc9e8b7f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:12.095502 27073 tablet_bootstrap.cc:654] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a88e92ebea134aecbf7fd8de8e29dae9: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:12.101960 27074 tablet_bootstrap.cc:492] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P 6a6db57534424adabd9ce75da8a7c4b3: Bootstrap starting.
I20260619 02:51:12.103590 27074 tablet_bootstrap.cc:654] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P 6a6db57534424adabd9ce75da8a7c4b3: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:12.107800 27073 tablet_bootstrap.cc:492] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a88e92ebea134aecbf7fd8de8e29dae9: No bootstrap required, opened a new log
I20260619 02:51:12.107995 27073 ts_tablet_manager.cc:1397] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a88e92ebea134aecbf7fd8de8e29dae9: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.006s
I20260619 02:51:12.108860 27073 raft_consensus.cc:348] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:12.109205 27073 raft_consensus.cc:374] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:12.109294 27073 raft_consensus.cc:729] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a88e92ebea134aecbf7fd8de8e29dae9, State: Initialized, Role: FOLLOWER
I20260619 02:51:12.109579 27073 consensus_queue.cc:260] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a88e92ebea134aecbf7fd8de8e29dae9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:12.110132 27073 ts_tablet_manager.cc:1428] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a88e92ebea134aecbf7fd8de8e29dae9: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 02:51:12.113804 27097 log.cc:1196] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-1/wal/wals/97db34710f924827b609603b3784b5b8
I20260619 02:51:12.114334 27097 ts_tablet_manager.cc:1939] T 97db34710f924827b609603b3784b5b8 P 6a6db57534424adabd9ce75da8a7c4b3: Deleting consensus metadata
I20260619 02:51:12.115942 26550 catalog_manager.cc:4841] TS 6a6db57534424adabd9ce75da8a7c4b3 (127.18.134.2:33961): tablet 97db34710f924827b609603b3784b5b8 (table test-table [id=4f0123882ec844f6af946947a4b44aad]) successfully deleted
I20260619 02:51:12.118999 27098 raft_consensus.cc:2229] T 97db34710f924827b609603b3784b5b8 P a88e92ebea134aecbf7fd8de8e29dae9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:51:12.119587 27098 pending_rounds.cc:62] T 97db34710f924827b609603b3784b5b8 P a88e92ebea134aecbf7fd8de8e29dae9: Trying to abort 2 pending ops.
I20260619 02:51:12.119699 27098 pending_rounds.cc:68] T 97db34710f924827b609603b3784b5b8 P a88e92ebea134aecbf7fd8de8e29dae9: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7298406285267369984 op_type: WRITE_OP write_request { tablet_id: "97db34710f924827b609603b3784b5b8" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\313\337\322\321\006\"8\n\ntest-admin\022*\n 4f0123882ec844f6af946947a4b44aad\020\001\030\001 \001(\001" signature: "<redacted>""\203v\235\007\244\025\344BO\223^2Z[vC\316?\253d\216f\r0a@\260\375\366\355<\240;\336\202W\336:W\365ss7\314\245\005/\223)\372\035\272\302\301p\255%~\345z\235\374_\001" signing_key_seq_num: 0 } } request_id { client_id: "1f591ca9e4f147239ed4dae38264ab55" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260619 02:51:12.120350 27098 pending_rounds.cc:68] T 97db34710f924827b609603b3784b5b8 P a88e92ebea134aecbf7fd8de8e29dae9: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7298406285053333504 op_type: NO_OP noop_request { }
I20260619 02:51:12.120494 27098 raft_consensus.cc:2866] T 97db34710f924827b609603b3784b5b8 P a88e92ebea134aecbf7fd8de8e29dae9 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260619 02:51:12.120630 27098 raft_consensus.cc:2258] T 97db34710f924827b609603b3784b5b8 P a88e92ebea134aecbf7fd8de8e29dae9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:51:12.121793 27099 ts_tablet_manager.cc:1905] T 97db34710f924827b609603b3784b5b8 P a1d869606d534bd488f4f667e355a57a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 02:51:12.128101 27152 consensus_queue.cc:1159] T 97db34710f924827b609603b3784b5b8 P a88e92ebea134aecbf7fd8de8e29dae9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20260619 02:51:12.128954 27098 ts_tablet_manager.cc:1905] T 97db34710f924827b609603b3784b5b8 P a88e92ebea134aecbf7fd8de8e29dae9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:51:12.129838 27099 ts_tablet_manager.cc:1918] T 97db34710f924827b609603b3784b5b8 P a1d869606d534bd488f4f667e355a57a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:51:12.130357 27099 log.cc:1196] T 97db34710f924827b609603b3784b5b8 P a1d869606d534bd488f4f667e355a57a: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-0/wal/wals/97db34710f924827b609603b3784b5b8
I20260619 02:51:12.131263 27099 ts_tablet_manager.cc:1939] T 97db34710f924827b609603b3784b5b8 P a1d869606d534bd488f4f667e355a57a: Deleting consensus metadata
I20260619 02:51:12.132380 26547 catalog_manager.cc:4841] TS a1d869606d534bd488f4f667e355a57a (127.18.134.1:33759): tablet 97db34710f924827b609603b3784b5b8 (table test-table [id=4f0123882ec844f6af946947a4b44aad]) successfully deleted
I20260619 02:51:12.135792 27098 ts_tablet_manager.cc:1918] T 97db34710f924827b609603b3784b5b8 P a88e92ebea134aecbf7fd8de8e29dae9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:51:12.135939 27098 log.cc:1196] T 97db34710f924827b609603b3784b5b8 P a88e92ebea134aecbf7fd8de8e29dae9: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-2/wal/wals/97db34710f924827b609603b3784b5b8
I20260619 02:51:12.136340 27098 ts_tablet_manager.cc:1939] T 97db34710f924827b609603b3784b5b8 P a88e92ebea134aecbf7fd8de8e29dae9: Deleting consensus metadata
I20260619 02:51:12.137406 26547 catalog_manager.cc:4841] TS a88e92ebea134aecbf7fd8de8e29dae9 (127.18.134.3:33887): tablet 97db34710f924827b609603b3784b5b8 (table test-table [id=4f0123882ec844f6af946947a4b44aad]) successfully deleted
I20260619 02:51:12.139313 27075 tablet_bootstrap.cc:492] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a: No bootstrap required, opened a new log
I20260619 02:51:12.139500 27075 ts_tablet_manager.cc:1397] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a: Time spent bootstrapping tablet: real 0.054s user 0.005s sys 0.006s
I20260619 02:51:12.140264 27075 raft_consensus.cc:348] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:12.140540 27075 raft_consensus.cc:374] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:12.140630 27075 raft_consensus.cc:729] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1d869606d534bd488f4f667e355a57a, State: Initialized, Role: FOLLOWER
I20260619 02:51:12.141791 27074 tablet_bootstrap.cc:492] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P 6a6db57534424adabd9ce75da8a7c4b3: No bootstrap required, opened a new log
I20260619 02:51:12.141969 27074 ts_tablet_manager.cc:1397] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P 6a6db57534424adabd9ce75da8a7c4b3: Time spent bootstrapping tablet: real 0.040s user 0.000s sys 0.009s
I20260619 02:51:12.142802 27074 raft_consensus.cc:348] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:12.143069 27074 raft_consensus.cc:374] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:12.143162 27074 raft_consensus.cc:729] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a6db57534424adabd9ce75da8a7c4b3, State: Initialized, Role: FOLLOWER
I20260619 02:51:12.143432 27074 consensus_queue.cc:260] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P 6a6db57534424adabd9ce75da8a7c4b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:12.146292 27074 ts_tablet_manager.cc:1428] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P 6a6db57534424adabd9ce75da8a7c4b3: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260619 02:51:12.148593 27075 consensus_queue.cc:260] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:12.149273 27075 ts_tablet_manager.cc:1428] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a: Time spent starting tablet: real 0.010s user 0.003s sys 0.000s
I20260619 02:51:12.187992 27081 raft_consensus.cc:482] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:51:12.188190 27081 raft_consensus.cc:504] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:12.188797 27081 leader_election.cc:288] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6a6db57534424adabd9ce75da8a7c4b3 (127.18.134.2:33961), a88e92ebea134aecbf7fd8de8e29dae9 (127.18.134.3:33887)
I20260619 02:51:12.189316 26862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cd4c91eaea5493db8b164bfc9e8b7f1" candidate_uuid: "a1d869606d534bd488f4f667e355a57a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a6db57534424adabd9ce75da8a7c4b3" is_pre_election: true
I20260619 02:51:12.189327 27002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cd4c91eaea5493db8b164bfc9e8b7f1" candidate_uuid: "a1d869606d534bd488f4f667e355a57a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" is_pre_election: true
I20260619 02:51:12.189553 26862 raft_consensus.cc:2454] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1d869606d534bd488f4f667e355a57a in term 0.
I20260619 02:51:12.189568 27002 raft_consensus.cc:2454] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1d869606d534bd488f4f667e355a57a in term 0.
I20260619 02:51:12.189977 26653 leader_election.cc:302] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a6db57534424adabd9ce75da8a7c4b3, a1d869606d534bd488f4f667e355a57a; no voters:
I20260619 02:51:12.190240 27081 raft_consensus.cc:2781] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:51:12.190344 27081 raft_consensus.cc:482] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:51:12.190428 27081 raft_consensus.cc:3037] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:12.192373 27081 raft_consensus.cc:504] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:12.192859 27081 leader_election.cc:288] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a [CANDIDATE]: Term 1 election: Requested vote from peers 6a6db57534424adabd9ce75da8a7c4b3 (127.18.134.2:33961), a88e92ebea134aecbf7fd8de8e29dae9 (127.18.134.3:33887)
I20260619 02:51:12.193301 26862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cd4c91eaea5493db8b164bfc9e8b7f1" candidate_uuid: "a1d869606d534bd488f4f667e355a57a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a6db57534424adabd9ce75da8a7c4b3"
I20260619 02:51:12.193388 27002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cd4c91eaea5493db8b164bfc9e8b7f1" candidate_uuid: "a1d869606d534bd488f4f667e355a57a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a88e92ebea134aecbf7fd8de8e29dae9"
I20260619 02:51:12.193473 26862 raft_consensus.cc:3037] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P 6a6db57534424adabd9ce75da8a7c4b3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:12.193589 27002 raft_consensus.cc:3037] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a88e92ebea134aecbf7fd8de8e29dae9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:12.195741 26862 raft_consensus.cc:2454] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P 6a6db57534424adabd9ce75da8a7c4b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1d869606d534bd488f4f667e355a57a in term 1.
I20260619 02:51:12.196106 26653 leader_election.cc:302] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a6db57534424adabd9ce75da8a7c4b3, a1d869606d534bd488f4f667e355a57a; no voters:
I20260619 02:51:12.196350 27081 raft_consensus.cc:2781] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:51:12.196430 27002 raft_consensus.cc:2454] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a88e92ebea134aecbf7fd8de8e29dae9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1d869606d534bd488f4f667e355a57a in term 1.
I20260619 02:51:12.196807 27081 raft_consensus.cc:686] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a [term 1 LEADER]: Becoming Leader. State: Replica: a1d869606d534bd488f4f667e355a57a, State: Running, Role: LEADER
I20260619 02:51:12.197291 27081 consensus_queue.cc:237] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } }
I20260619 02:51:12.199625 26563 catalog_manager.cc:5495] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a reported cstate change: term changed from 0 to 1, leader changed from <none> to a1d869606d534bd488f4f667e355a57a (127.18.134.1). New cstate: current_term: 1 leader_uuid: "a1d869606d534bd488f4f667e355a57a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1d869606d534bd488f4f667e355a57a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33759 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 } health_report { overall_health: UNKNOWN } } }
I20260619 02:51:12.239679 27072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:12.234441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34036 (local address 127.18.134.1:33759)
0619 02:51:12.234586 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:12.234592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:12.234779 (+ 187us) server_negotiation.cc:407] Connection header received
0619 02:51:12.234967 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:12.234971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:12.235040 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:12.235109 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:51:12.235464 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:12.236205 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:12.238080 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:12.238469 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:12.238558 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:12.239020 (+ 462us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:51:12.239129 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:51:12.239198 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:12.239331 (+ 133us) server_negotiation.cc:299] Negotiation successful
0619 02:51:12.239391 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 02:51:12.252936 27072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:12.247635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:47961 (local address 127.18.134.1:33759)
0619 02:51:12.247848 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:12.247855 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:12.248168 (+ 313us) server_negotiation.cc:407] Connection header received
0619 02:51:12.248296 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:12.248302 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:12.248378 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:12.248461 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:51:12.248932 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:12.249593 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:12.251381 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:12.251814 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:12.251904 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:12.252362 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:51:12.252452 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:51:12.252521 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:12.252640 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 02:51:12.252707 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 02:51:12.253180 27164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:12.247376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:33759 (local address 127.18.134.1:47961)
0619 02:51:12.248084 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:12.248099 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:12.248201 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:12.248648 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:12.248653 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:12.248664 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 02:51:12.248806 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:12.248816 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:12.249717 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:12.249722 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:12.251255 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:12.251268 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:12.251973 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:12.251980 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:12.252161 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:12.252207 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 02:51:12.252537 (+ 330us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 02:51:12.252547 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 02:51:12.252623 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 02:51:12.252756 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":464,"threads_started":1}
I20260619 02:51:12.260706 26862 raft_consensus.cc:1261] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P 6a6db57534424adabd9ce75da8a7c4b3 [term 1 FOLLOWER]: Refusing update from remote peer a1d869606d534bd488f4f667e355a57a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:51:12.260859 27002 raft_consensus.cc:1261] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a88e92ebea134aecbf7fd8de8e29dae9 [term 1 FOLLOWER]: Refusing update from remote peer a1d869606d534bd488f4f667e355a57a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:51:12.261219 27160 consensus_queue.cc:1035] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a6db57534424adabd9ce75da8a7c4b3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 33961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:51:12.261490 27081 consensus_queue.cc:1035] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a88e92ebea134aecbf7fd8de8e29dae9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 33887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:51:12.322592 26563 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:51:12.322861 26563 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:51:12.325466 26563 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de7f10a123774146b91cb4077e5aebb1: Sending DeleteTablet for 3 replicas of tablet 6cd4c91eaea5493db8b164bfc9e8b7f1
I20260619 02:51:12.326261 26839 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cd4c91eaea5493db8b164bfc9e8b7f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:51:12 UTC) from {username='oryx', principal='oryx/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:48246
I20260619 02:51:12.326552 27097 tablet_replica.cc:330] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P 6a6db57534424adabd9ce75da8a7c4b3: stopping tablet replica
I20260619 02:51:12.326623 26978 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cd4c91eaea5493db8b164bfc9e8b7f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:51:12 UTC) from {username='oryx', principal='oryx/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:57604
I20260619 02:51:12.326784 27097 raft_consensus.cc:2229] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P 6a6db57534424adabd9ce75da8a7c4b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:51:12.326887 27098 tablet_replica.cc:330] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a88e92ebea134aecbf7fd8de8e29dae9: stopping tablet replica
I20260619 02:51:12.326978 27097 raft_consensus.cc:2258] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P 6a6db57534424adabd9ce75da8a7c4b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:51:12.327044 27098 raft_consensus.cc:2229] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a88e92ebea134aecbf7fd8de8e29dae9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:51:12.327117 26700 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cd4c91eaea5493db8b164bfc9e8b7f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:51:12 UTC) from {username='oryx', principal='oryx/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:34020
I20260619 02:51:12.327212 27098 raft_consensus.cc:2258] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a88e92ebea134aecbf7fd8de8e29dae9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:51:12.327373 27099 tablet_replica.cc:330] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a: stopping tablet replica
I20260619 02:51:12.327554 27099 raft_consensus.cc:2229] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:51:12.327782 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 26634
I20260619 02:51:12.327863 27099 raft_consensus.cc:2258] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:51:12.329717 27098 ts_tablet_manager.cc:1905] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a88e92ebea134aecbf7fd8de8e29dae9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:51:12.329751 27097 ts_tablet_manager.cc:1905] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P 6a6db57534424adabd9ce75da8a7c4b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:51:12.330794 27099 ts_tablet_manager.cc:1905] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:51:12.336339 27098 ts_tablet_manager.cc:1918] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a88e92ebea134aecbf7fd8de8e29dae9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:51:12.336364 27097 ts_tablet_manager.cc:1918] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P 6a6db57534424adabd9ce75da8a7c4b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:51:12.336508 27097 log.cc:1196] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P 6a6db57534424adabd9ce75da8a7c4b3: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-1/wal/wals/6cd4c91eaea5493db8b164bfc9e8b7f1
I20260619 02:51:12.336629 27098 log.cc:1196] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a88e92ebea134aecbf7fd8de8e29dae9: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-2/wal/wals/6cd4c91eaea5493db8b164bfc9e8b7f1
I20260619 02:51:12.336946 27097 ts_tablet_manager.cc:1939] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P 6a6db57534424adabd9ce75da8a7c4b3: Deleting consensus metadata
I20260619 02:51:12.337086 27098 ts_tablet_manager.cc:1939] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a88e92ebea134aecbf7fd8de8e29dae9: Deleting consensus metadata
I20260619 02:51:12.337980 26550 catalog_manager.cc:4841] TS 6a6db57534424adabd9ce75da8a7c4b3 (127.18.134.2:33961): tablet 6cd4c91eaea5493db8b164bfc9e8b7f1 (table test-table [id=4fc7b305166c4980a812941a958562cd]) successfully deleted
I20260619 02:51:12.338163 26547 catalog_manager.cc:4841] TS a88e92ebea134aecbf7fd8de8e29dae9 (127.18.134.3:33887): tablet 6cd4c91eaea5493db8b164bfc9e8b7f1 (table test-table [id=4fc7b305166c4980a812941a958562cd]) successfully deleted
I20260619 02:51:12.339139 27099 ts_tablet_manager.cc:1918] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 02:51:12.339293 27099 log.cc:1196] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781837410515220-18968-0/minicluster-data/ts-0/wal/wals/6cd4c91eaea5493db8b164bfc9e8b7f1
I20260619 02:51:12.339696 27099 ts_tablet_manager.cc:1939] T 6cd4c91eaea5493db8b164bfc9e8b7f1 P a1d869606d534bd488f4f667e355a57a: Deleting consensus metadata
I20260619 02:51:12.340701 26547 catalog_manager.cc:4841] TS a1d869606d534bd488f4f667e355a57a (127.18.134.1:33759): tablet 6cd4c91eaea5493db8b164bfc9e8b7f1 (table test-table [id=4fc7b305166c4980a812941a958562cd]) successfully deleted
Jun 19 02:51:12 dist-test-slave-9mr0 krb5kdc[26518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.134.1@KRBTEST.COM
I20260619 02:51:12.356215 27072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:12.334841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34048 (local address 127.18.134.1:33759)
0619 02:51:12.335004 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:12.335011 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:12.340996 (+ 5985us) server_negotiation.cc:407] Connection header received
0619 02:51:12.341166 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:12.341169 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:12.341228 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:12.341303 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:12.342816 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:12.343602 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:12.345313 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:12.345722 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:12.345824 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:12.350519 (+ 4695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:12.350532 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:12.350549 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:12.350591 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:12.353542 (+ 2951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:12.354336 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:12.354343 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:12.354346 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:12.354481 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:12.354888 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:12.354891 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:12.354893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:12.355255 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:12.355492 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:12.355789 (+ 297us) server_negotiation.cc:299] Negotiation successful
0619 02:51:12.355946 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 02:51:12.356868 26765 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:12.519255 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 26634
I20260619 02:51:12.540897 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 26774
Jun 19 02:51:12 dist-test-slave-9mr0 krb5kdc[26518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.134.2@KRBTEST.COM
I20260619 02:51:12.559466 27170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:12.541620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48260 (local address 127.18.134.2:33961)
0619 02:51:12.542455 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:12.542462 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:12.542482 (+ 20us) server_negotiation.cc:407] Connection header received
0619 02:51:12.542549 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:12.542554 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:12.542669 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:12.542787 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:12.543954 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:12.545194 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:12.546760 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:12.547445 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:12.547673 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:12.552232 (+ 4559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:12.552250 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:12.552269 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:12.552334 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:12.556556 (+ 4222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:12.557293 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:12.557302 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:12.557305 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:12.557460 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:12.557790 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:12.557795 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:12.557798 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:12.558268 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:12.558550 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:12.558816 (+ 266us) server_negotiation.cc:299] Negotiation successful
0619 02:51:12.559044 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":492,"threads_started":1}
I20260619 02:51:12.560207 26904 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:12.687310 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 26774
I20260619 02:51:12.704330 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 26913
Jun 19 02:51:12 dist-test-slave-9mr0 krb5kdc[26518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.134.3@KRBTEST.COM
I20260619 02:51:12.720176 27172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:12.705070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57628 (local address 127.18.134.3:33887)
0619 02:51:12.705524 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:12.705529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:12.705547 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:51:12.705600 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:12.705604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:12.705671 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:12.705765 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:12.707154 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:12.707862 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:12.709190 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:12.709639 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:12.709744 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:12.714213 (+ 4469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:12.714229 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:12.714244 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:12.714287 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:12.717057 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:12.717882 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:12.717890 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:12.717892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:12.718053 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:12.718483 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:12.718487 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:12.718489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:12.719010 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:12.719273 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:12.719593 (+ 320us) server_negotiation.cc:299] Negotiation successful
0619 02:51:12.719777 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":249,"threads_started":1}
I20260619 02:51:12.721004 27043 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:12.840619 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 26913
I20260619 02:51:12.857244 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 26531
I20260619 02:51:12.858428 26603 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:12.969699 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 26531
2026-06-19T02:51:12Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6319 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:51:13 dist-test-slave-9mr0 krb5kdc[27177](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:51:13 dist-test-slave-9mr0 krb5kdc[27177](info): set up 2 sockets
Jun 19 02:51:13 dist-test-slave-9mr0 krb5kdc[27177](info): commencing operation
krb5kdc: starting...
W20260619 02:51:14.015408 18968 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:51:14 dist-test-slave-9mr0 krb5kdc[27177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:51:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:51:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:51:14.224148 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:37109
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:43907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:37109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:51:14.541579 27190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:14.541865 27190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:14.541994 27190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:14.549171 27190 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:51:14.549283 27190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:14.549353 27190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:14.549430 27190 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:51:14.549485 27190 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:51:14.559356 27190 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:43907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:37109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:37109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:14.560623 27190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:14.561933 27190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:14.572944 27195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:14.572983 27198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:14.573559 27196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:14.573777 27190 server_base.cc:1034] running on GCE node
I20260619 02:51:14.574483 27190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:14.575934 27190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:14.577160 27190 hybrid_clock.cc:648] HybridClock initialized: now 1781837474577081 us; error 75 us; skew 500 ppm
I20260619 02:51:14.577486 27190 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-19T02:51:14Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1674 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:51:14 dist-test-slave-9mr0 krb5kdc[27204](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:51:14 dist-test-slave-9mr0 krb5kdc[27204](info): set up 2 sockets
Jun 19 02:51:14 dist-test-slave-9mr0 krb5kdc[27204](info): commencing operation
krb5kdc: starting...
W20260619 02:51:15.651546 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.967s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:51:15 dist-test-slave-9mr0 krb5kdc[27204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:51:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:51:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:51:15.858927 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:39757
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:33407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:39757
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:51:16.171278 27217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:16.171571 27217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:16.171697 27217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:16.178797 27217 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:51:16.178900 27217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:16.178967 27217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:16.179049 27217 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:51:16.179104 27217 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:51:16.188849 27217 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:33407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:39757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:39757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:16.190110 27217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:16.191386 27217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:16.203172 27222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:16.203220 27223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:16.203241 27225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:16.203961 27217 server_base.cc:1034] running on GCE node
I20260619 02:51:16.204689 27217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:16.206096 27217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:16.207316 27217 hybrid_clock.cc:648] HybridClock initialized: now 1781837476207265 us; error 39 us; skew 500 ppm
I20260619 02:51:16.207659 27217 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:51:16 dist-test-slave-9mr0 krb5kdc[27204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837476, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:51:16.210649 27217 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:51:16.212011 27217 webserver.cc:460] Webserver started at http://127.18.134.62:33257/ using document root <none> and password file <none>
I20260619 02:51:16.212651 27217 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:16.212749 27217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:16.213074 27217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:51:16.215996 27217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "4ab8bcdacc8a4709b322cc9a994e4a0e"
format_stamp: "Formatted at 2026-06-19 02:51:16 on dist-test-slave-9mr0"
server_key: "cd9f030c6fe6deb07dcbe085b80e4094"
server_key_iv: "94ab60c0682b9c5219b82c4978871ec9"
server_key_version: "encryptionkey@0"
I20260619 02:51:16.216699 27217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "4ab8bcdacc8a4709b322cc9a994e4a0e"
format_stamp: "Formatted at 2026-06-19 02:51:16 on dist-test-slave-9mr0"
server_key: "cd9f030c6fe6deb07dcbe085b80e4094"
server_key_iv: "94ab60c0682b9c5219b82c4978871ec9"
server_key_version: "encryptionkey@0"
I20260619 02:51:16.222066 27217 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260619 02:51:16.225699 27232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:16.227272 27217 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:51:16.227448 27217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "4ab8bcdacc8a4709b322cc9a994e4a0e"
format_stamp: "Formatted at 2026-06-19 02:51:16 on dist-test-slave-9mr0"
server_key: "cd9f030c6fe6deb07dcbe085b80e4094"
server_key_iv: "94ab60c0682b9c5219b82c4978871ec9"
server_key_version: "encryptionkey@0"
I20260619 02:51:16.227666 27217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:16.243415 27217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:16.247606 27217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:16.247869 27217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:16.264740 27217 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:39757
I20260619 02:51:16.264747 27283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:39757 every 8 connection(s)
I20260619 02:51:16.266065 27217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:51:16.268177 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 27217
I20260619 02:51:16.268380 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:51:16.268985 18968 external_mini_cluster.cc:1428] Setting key e7b5292645ccf49a57e1caaf92246abe
I20260619 02:51:16.269788 27284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 19 02:51:16 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:51:16.283174 27284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e: Bootstrap starting.
I20260619 02:51:16.286900 27284 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:16.288048 27284 log.cc:824] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:51:16.291262 27284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e: No bootstrap required, opened a new log
I20260619 02:51:16.291908 27287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:16.270814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45790 (local address 127.18.134.62:39757)
0619 02:51:16.271623 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:16.271637 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:16.271685 (+ 48us) server_negotiation.cc:407] Connection header received
0619 02:51:16.273289 (+ 1604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:16.273317 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:16.273825 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:16.274244 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:16.275583 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:16.277539 (+ 1956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:16.278835 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:16.279819 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:16.279980 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:16.283910 (+ 3930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:16.283920 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:16.283936 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:16.283985 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:16.287554 (+ 3569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:16.288229 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:16.288237 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:16.288244 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:16.288430 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:16.288924 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:16.288928 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:16.288930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:16.289543 (+ 613us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:16.289847 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:16.290218 (+ 371us) server_negotiation.cc:299] Negotiation successful
0619 02:51:16.290614 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":337,"threads_started":1}
I20260619 02:51:16.299219 27284 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab8bcdacc8a4709b322cc9a994e4a0e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39757 } }
I20260619 02:51:16.299551 27284 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:16.299623 27284 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ab8bcdacc8a4709b322cc9a994e4a0e, State: Initialized, Role: FOLLOWER
I20260619 02:51:16.300136 27284 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab8bcdacc8a4709b322cc9a994e4a0e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39757 } }
I20260619 02:51:16.300334 27284 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:51:16.300426 27284 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:51:16.300549 27284 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:16.302595 27284 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab8bcdacc8a4709b322cc9a994e4a0e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39757 } }
I20260619 02:51:16.303131 27284 leader_election.cc:302] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ab8bcdacc8a4709b322cc9a994e4a0e; no voters:
I20260619 02:51:16.303726 27284 leader_election.cc:288] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:51:16.303835 27289 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:51:16.304594 27289 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [term 1 LEADER]: Becoming Leader. State: Replica: 4ab8bcdacc8a4709b322cc9a994e4a0e, State: Running, Role: LEADER
I20260619 02:51:16.305074 27289 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab8bcdacc8a4709b322cc9a994e4a0e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39757 } }
I20260619 02:51:16.305629 27284 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:51:16.307706 27290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ab8bcdacc8a4709b322cc9a994e4a0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab8bcdacc8a4709b322cc9a994e4a0e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39757 } } }
I20260619 02:51:16.308079 27290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [sys.catalog]: This master's current role is: LEADER
I20260619 02:51:16.309450 27291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ab8bcdacc8a4709b322cc9a994e4a0e. Latest consensus state: current_term: 1 leader_uuid: "4ab8bcdacc8a4709b322cc9a994e4a0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab8bcdacc8a4709b322cc9a994e4a0e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39757 } } }
I20260619 02:51:16.309686 27291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [sys.catalog]: This master's current role is: LEADER
I20260619 02:51:16.310209 27297 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:51:16.314260 27297 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:51:16.321988 27297 catalog_manager.cc:1347] Generated new cluster ID: 299af98b6cec4579b05626a572a2b38b
I20260619 02:51:16.322106 27297 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:51:16.357796 27297 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:51:16.359251 27297 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:51:16.366921 27297 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e: Generated new TSK 0
I20260619 02:51:16.367717 27297 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.134.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
I20260619 02:51:16.419101 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:0
--local_ip_for_outbound_sockets=127.18.134.1
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:39757
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:33407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:51:16.730646 27310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:16.730933 27310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:16.731055 27310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:16.738286 27310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:16.738397 27310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:16.738550 27310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:51:16.748185 27310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:33407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:39757
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:16.749442 27310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:16.750713 27310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:16.763171 27315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:16.763171 27316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:16.763182 27318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:16.763991 27310 server_base.cc:1034] running on GCE node
I20260619 02:51:16.764557 27310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:16.765445 27310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:16.766677 27310 hybrid_clock.cc:648] HybridClock initialized: now 1781837476766604 us; error 34 us; skew 500 ppm
I20260619 02:51:16.766993 27310 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:51:16 dist-test-slave-9mr0 krb5kdc[27204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837476, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:51:16.770015 27310 init.cc:377] Logged in from keytab as kudu/127.18.134.1@KRBTEST.COM (short username kudu)
I20260619 02:51:16.771485 27310 webserver.cc:460] Webserver started at http://127.18.134.1:33807/ using document root <none> and password file <none>
I20260619 02:51:16.772136 27310 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:16.772238 27310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:16.772523 27310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:51:16.775329 27310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/data/instance:
uuid: "90bc8507baac465fa1945015c2694378"
format_stamp: "Formatted at 2026-06-19 02:51:16 on dist-test-slave-9mr0"
server_key: "dd2f3ba0d67b0c182e08ead7d6d9c7f7"
server_key_iv: "da05da27e6a36f585367da2eb2a96cb5"
server_key_version: "encryptionkey@0"
I20260619 02:51:16.776067 27310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance:
uuid: "90bc8507baac465fa1945015c2694378"
format_stamp: "Formatted at 2026-06-19 02:51:16 on dist-test-slave-9mr0"
server_key: "dd2f3ba0d67b0c182e08ead7d6d9c7f7"
server_key_iv: "da05da27e6a36f585367da2eb2a96cb5"
server_key_version: "encryptionkey@0"
I20260619 02:51:16.781086 27310 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:51:16.784471 27325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:16.785840 27310 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 02:51:16.786001 27310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "90bc8507baac465fa1945015c2694378"
format_stamp: "Formatted at 2026-06-19 02:51:16 on dist-test-slave-9mr0"
server_key: "dd2f3ba0d67b0c182e08ead7d6d9c7f7"
server_key_iv: "da05da27e6a36f585367da2eb2a96cb5"
server_key_version: "encryptionkey@0"
I20260619 02:51:16.786180 27310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:16.804309 27310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:16.808838 27310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:16.809108 27310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:16.810211 27310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:51:16.811582 27310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:51:16.811651 27310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:16.811765 27310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:51:16.811813 27310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:16.843273 27310 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:46743
I20260619 02:51:16.843327 27437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:46743 every 8 connection(s)
I20260619 02:51:16.844507 27310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:51:16.848441 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 27310
I20260619 02:51:16.848639 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance
I20260619 02:51:16.849413 18968 external_mini_cluster.cc:1428] Setting key f705118afc5126320422c0fdfcf3eddd
Jun 19 02:51:16 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837476, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:51:16.866295 27440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:16.846920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:38715 (local address 127.18.134.62:39757)
0619 02:51:16.847356 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:16.847361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:16.848570 (+ 1209us) server_negotiation.cc:407] Connection header received
0619 02:51:16.850733 (+ 2163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:16.850739 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:16.850809 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:16.850921 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:16.852808 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:16.853506 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:16.855495 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:16.855914 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:16.856006 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:16.859991 (+ 3985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:16.859997 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:16.860001 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:16.860039 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:16.863188 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:16.863967 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:16.863973 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:16.863977 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:16.864121 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:16.864489 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:16.864493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:16.864496 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:16.864819 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:16.864973 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:16.865666 (+ 693us) server_negotiation.cc:299] Negotiation successful
0619 02:51:16.865886 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":234,"threads_started":1}
I20260619 02:51:16.867238 27441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:16.847510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.1:38715)
0619 02:51:16.848370 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:16.848426 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:16.850465 (+ 2039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:16.851167 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:16.851175 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:16.851856 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:16.852630 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:16.852654 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:16.853647 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:16.853652 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:16.855359 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:16.855372 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:16.856121 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:16.856127 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:16.856421 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:16.857118 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:16.857149 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:16.859752 (+ 2603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:16.863363 (+ 3611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:16.863371 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:16.863387 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:16.863843 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:16.864211 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:16.864214 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:16.864216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:16.864393 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:16.864960 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:16.864967 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:16.865319 (+ 352us) client_negotiation.cc:769] Sending connection context
0619 02:51:16.865549 (+ 230us) client_negotiation.cc:240] Negotiation successful
0619 02:51:16.866011 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":340,"threads_started":1}
I20260619 02:51:16.868973 27438 heartbeater.cc:347] Connected to a master server at 127.18.134.62:39757
I20260619 02:51:16.869323 27438 heartbeater.cc:464] Registering TS with master...
I20260619 02:51:16.870270 27438 heartbeater.cc:511] Master 127.18.134.62:39757 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.134.2@KRBTEST.COM; defaulting to no policy
I20260619 02:51:16.872606 27249 ts_manager.cc:194] Registered new tserver with Master: 90bc8507baac465fa1945015c2694378 (127.18.134.1:46743)
I20260619 02:51:16.874099 27249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.1@KRBTEST.COM'} at 127.18.134.1:38715
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
I20260619 02:51:16.899902 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:0
--local_ip_for_outbound_sockets=127.18.134.2
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:39757
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:33407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:51:17.224247 27444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:17.224537 27444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:17.224656 27444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:17.232105 27444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:17.232227 27444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:17.232373 27444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:51:17.242163 27444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:33407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:39757
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:17.243525 27444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:17.244800 27444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:17.257613 27449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:17.257665 27452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:17.257642 27450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:17.259011 27444 server_base.cc:1034] running on GCE node
I20260619 02:51:17.259639 27444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:17.260568 27444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:17.261762 27444 hybrid_clock.cc:648] HybridClock initialized: now 1781837477261713 us; error 46 us; skew 500 ppm
I20260619 02:51:17.262090 27444 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:51:17 dist-test-slave-9mr0 krb5kdc[27204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837477, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:51:17.265544 27444 init.cc:377] Logged in from keytab as kudu/127.18.134.2@KRBTEST.COM (short username kudu)
I20260619 02:51:17.267427 27444 webserver.cc:460] Webserver started at http://127.18.134.2:38375/ using document root <none> and password file <none>
I20260619 02:51:17.268326 27444 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:17.268455 27444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:17.268787 27444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:51:17.272795 27444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/data/instance:
uuid: "4d2275694b614a7195cf2d9659ff4217"
format_stamp: "Formatted at 2026-06-19 02:51:17 on dist-test-slave-9mr0"
server_key: "6fc0b9a83ad33fd81db4e211b8a2a4c1"
server_key_iv: "91361ecbc8d5256fca67704f1b73f246"
server_key_version: "encryptionkey@0"
I20260619 02:51:17.273869 27444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance:
uuid: "4d2275694b614a7195cf2d9659ff4217"
format_stamp: "Formatted at 2026-06-19 02:51:17 on dist-test-slave-9mr0"
server_key: "6fc0b9a83ad33fd81db4e211b8a2a4c1"
server_key_iv: "91361ecbc8d5256fca67704f1b73f246"
server_key_version: "encryptionkey@0"
I20260619 02:51:17.281392 27444 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260619 02:51:17.285745 27459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:17.287249 27444 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 02:51:17.287390 27444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "4d2275694b614a7195cf2d9659ff4217"
format_stamp: "Formatted at 2026-06-19 02:51:17 on dist-test-slave-9mr0"
server_key: "6fc0b9a83ad33fd81db4e211b8a2a4c1"
server_key_iv: "91361ecbc8d5256fca67704f1b73f246"
server_key_version: "encryptionkey@0"
I20260619 02:51:17.287560 27444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:17.299925 27444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:17.304059 27444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:17.304294 27444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:17.305159 27444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:51:17.306458 27444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:51:17.306529 27444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:17.306648 27444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:51:17.306690 27444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260619 02:51:17.337996 27444 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:39677
I20260619 02:51:17.338054 27571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:39677 every 8 connection(s)
I20260619 02:51:17.339507 27444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:51:17.341255 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 27444
I20260619 02:51:17.341427 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance
I20260619 02:51:17.341910 18968 external_mini_cluster.cc:1428] Setting key 45ea938210f915f2379ec83b92888eeb
Jun 19 02:51:17 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837477, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:51:17.362066 27440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:17.342126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:48749 (local address 127.18.134.62:39757)
0619 02:51:17.342329 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:17.342335 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:17.343648 (+ 1313us) server_negotiation.cc:407] Connection header received
0619 02:51:17.345454 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:17.345458 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:17.345549 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:17.345633 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:17.347963 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:17.348688 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:17.351165 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:17.351607 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:17.351713 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:17.355560 (+ 3847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:17.355566 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:17.355569 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:17.355604 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:17.358242 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:17.359167 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:17.359174 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:17.359178 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:17.359351 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:17.359820 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:17.359825 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:17.359828 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:17.360186 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:17.360355 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:17.361456 (+ 1101us) server_negotiation.cc:299] Negotiation successful
0619 02:51:17.361696 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
WARNING: no policy specified for kudu/127.18.134.3@KRBTEST.COM; defaulting to no policy
I20260619 02:51:17.362998 27574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:17.342618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.2:48749)
0619 02:51:17.343507 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:17.343560 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:17.345180 (+ 1620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:17.345911 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:17.345923 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:17.347120 (+ 1197us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:17.347770 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:17.347790 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:17.348894 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:17.348900 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:17.350986 (+ 2086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:17.351011 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:17.351829 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:17.351836 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:17.352127 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:17.352843 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:17.352869 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:17.355310 (+ 2441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:17.358442 (+ 3132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:17.358453 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:17.358475 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:17.358999 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:17.359490 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:17.359494 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:17.359497 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:17.359698 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:17.360440 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:17.360450 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:17.361037 (+ 587us) client_negotiation.cc:769] Sending connection context
0619 02:51:17.361297 (+ 260us) client_negotiation.cc:240] Negotiation successful
0619 02:51:17.361771 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":342,"threads_started":1}
I20260619 02:51:17.364475 27572 heartbeater.cc:347] Connected to a master server at 127.18.134.62:39757
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.3@KRBTEST.COM" created.
I20260619 02:51:17.364814 27572 heartbeater.cc:464] Registering TS with master...
I20260619 02:51:17.365515 27572 heartbeater.cc:511] Master 127.18.134.62:39757 requested a full tablet report, sending...
I20260619 02:51:17.367084 27249 ts_manager.cc:194] Registered new tserver with Master: 4d2275694b614a7195cf2d9659ff4217 (127.18.134.2:39677)
I20260619 02:51:17.368039 27249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.2@KRBTEST.COM'} at 127.18.134.2:48749
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
I20260619 02:51:17.390081 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:0
--local_ip_for_outbound_sockets=127.18.134.3
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:39757
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:33407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:51:17.707448 27577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:17.707760 27577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:17.707856 27577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:17.714910 27577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:17.715018 27577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:17.715124 27577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:51:17.724779 27577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:33407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:39757
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:17.726110 27577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:17.727473 27577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:17.741267 27582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:17.741228 27585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:17.741364 27577 server_base.cc:1034] running on GCE node
W20260619 02:51:17.741339 27583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:17.742218 27577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:17.743136 27577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:17.744334 27577 hybrid_clock.cc:648] HybridClock initialized: now 1781837477744283 us; error 41 us; skew 500 ppm
I20260619 02:51:17.744628 27577 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:51:17 dist-test-slave-9mr0 krb5kdc[27204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837477, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:51:17.747484 27577 init.cc:377] Logged in from keytab as kudu/127.18.134.3@KRBTEST.COM (short username kudu)
I20260619 02:51:17.748791 27577 webserver.cc:460] Webserver started at http://127.18.134.3:40427/ using document root <none> and password file <none>
I20260619 02:51:17.749413 27577 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:17.749531 27577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:17.749804 27577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:51:17.752533 27577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/data/instance:
uuid: "36b8df26080949bba15dd77661caf470"
format_stamp: "Formatted at 2026-06-19 02:51:17 on dist-test-slave-9mr0"
server_key: "5ad2538f67f597c549cd3e434993a40c"
server_key_iv: "fb3ae2bc6e4de28acfb02e9b99de04d6"
server_key_version: "encryptionkey@0"
I20260619 02:51:17.753228 27577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance:
uuid: "36b8df26080949bba15dd77661caf470"
format_stamp: "Formatted at 2026-06-19 02:51:17 on dist-test-slave-9mr0"
server_key: "5ad2538f67f597c549cd3e434993a40c"
server_key_iv: "fb3ae2bc6e4de28acfb02e9b99de04d6"
server_key_version: "encryptionkey@0"
I20260619 02:51:17.758035 27577 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260619 02:51:17.761426 27592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:17.762773 27577 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:51:17.762916 27577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "36b8df26080949bba15dd77661caf470"
format_stamp: "Formatted at 2026-06-19 02:51:17 on dist-test-slave-9mr0"
server_key: "5ad2538f67f597c549cd3e434993a40c"
server_key_iv: "fb3ae2bc6e4de28acfb02e9b99de04d6"
server_key_version: "encryptionkey@0"
I20260619 02:51:17.763087 27577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:17.781134 27577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:17.785127 27577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:17.785405 27577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:17.786303 27577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:51:17.787642 27577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:51:17.787724 27577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:17.787854 27577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:51:17.787906 27577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:17.818128 27577 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:34549
I20260619 02:51:17.818177 27704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:34549 every 8 connection(s)
I20260619 02:51:17.819761 27577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:51:17.829767 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 27577
I20260619 02:51:17.829941 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance
I20260619 02:51:17.830360 18968 external_mini_cluster.cc:1428] Setting key 70f879a54ddfbdef63e7146963b98e26
Jun 19 02:51:17 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837477, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:51:17.839079 27440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:17.822117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:56953 (local address 127.18.134.62:39757)
0619 02:51:17.822297 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:17.822301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:17.823548 (+ 1247us) server_negotiation.cc:407] Connection header received
0619 02:51:17.825274 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:17.825280 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:17.825385 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:17.825474 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:17.826960 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:17.827666 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:17.829155 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:17.829673 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:17.829805 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:17.833809 (+ 4004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:17.833813 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:17.833817 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:17.833850 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:17.836468 (+ 2618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:17.837154 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:17.837157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:17.837159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:17.837264 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:17.837640 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:17.837644 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:17.837646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:17.837933 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:17.838043 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:17.838664 (+ 621us) server_negotiation.cc:299] Negotiation successful
0619 02:51:17.838826 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 02:51:17.839990 27707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:17.822620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.3:56953)
0619 02:51:17.823355 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:17.823428 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:17.825015 (+ 1587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:17.825714 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:17.825722 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:17.826207 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:17.826799 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:17.826818 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:17.827817 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:17.827821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:17.829019 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:17.829031 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:17.829938 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:17.829946 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:17.830199 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:17.830906 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:17.830943 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:17.833619 (+ 2676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:17.836619 (+ 3000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:17.836626 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:17.836638 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:17.837063 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:17.837338 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:17.837341 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:17.837343 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:17.837506 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:17.838025 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:17.838031 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:17.838319 (+ 288us) client_negotiation.cc:769] Sending connection context
0619 02:51:17.838529 (+ 210us) client_negotiation.cc:240] Negotiation successful
0619 02:51:17.838968 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":279,"threads_started":1}
I20260619 02:51:17.841182 27705 heartbeater.cc:347] Connected to a master server at 127.18.134.62:39757
I20260619 02:51:17.841444 27705 heartbeater.cc:464] Registering TS with master...
I20260619 02:51:17.842134 27705 heartbeater.cc:511] Master 127.18.134.62:39757 requested a full tablet report, sending...
I20260619 02:51:17.843634 27249 ts_manager.cc:194] Registered new tserver with Master: 36b8df26080949bba15dd77661caf470 (127.18.134.3:34549)
I20260619 02:51:17.844640 27249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.3@KRBTEST.COM'} at 127.18.134.3:56953
I20260619 02:51:17.845674 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.134.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.134.1@KRBTEST.COM" created.
I20260619 02:51:17.877177 27438 heartbeater.cc:503] Master 127.18.134.62:39757 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.134.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.1.keytab.
Entry for principal oryx/127.18.134.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.1.keytab.
WARNING: no policy specified for oryx/127.18.134.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.134.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.134.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.2.keytab.
Entry for principal oryx/127.18.134.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.2.keytab.
WARNING: no policy specified for oryx/127.18.134.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.134.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.134.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.3.keytab.
Entry for principal oryx/127.18.134.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.3.keytab.
I20260619 02:51:17.965466 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 27310
Jun 19 02:51:17 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.1@KRBTEST.COM
I20260619 02:51:17.981452 27715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:17.966537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48182 (local address 127.18.134.1:46743)
0619 02:51:17.967392 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:17.967399 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:17.967424 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:51:17.967481 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:17.967488 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:17.967703 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:17.967825 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:17.969148 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:17.970176 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:17.971579 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:17.972131 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:17.972233 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:17.975986 (+ 3753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:17.975999 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:17.976014 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:17.976057 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:17.978907 (+ 2850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:17.979579 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:17.979586 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:17.979589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:17.979712 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:17.980035 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:17.980038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:17.980040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:17.980361 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:17.980598 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:17.980935 (+ 337us) server_negotiation.cc:299] Negotiation successful
0619 02:51:17.981135 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":709,"thread_start_us":556,"threads_started":1}
I20260619 02:51:17.982774 27433 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:18.063107 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 27310
I20260619 02:51:18.075731 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 27444
Jun 19 02:51:18 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.2@KRBTEST.COM
I20260619 02:51:18.090476 27717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:18.076656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37310 (local address 127.18.134.2:39677)
0619 02:51:18.077109 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:18.077117 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:18.077139 (+ 22us) server_negotiation.cc:407] Connection header received
0619 02:51:18.077199 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:18.077205 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:18.077401 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:18.077522 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:18.078686 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:18.079648 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:18.080835 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:18.081346 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:18.081437 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:18.085305 (+ 3868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:18.085317 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:18.085330 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:18.085371 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:18.088030 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:18.088594 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:18.088600 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:18.088602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:18.088727 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:18.089093 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:18.089097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:18.089099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:18.089423 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:18.089645 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:18.089903 (+ 258us) server_negotiation.cc:299] Negotiation successful
0619 02:51:18.090111 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":210,"threads_started":1}
I20260619 02:51:18.091758 27567 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:18.171998 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 27444
I20260619 02:51:18.184265 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 27577
Jun 19 02:51:18 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.3@KRBTEST.COM
I20260619 02:51:18.200104 27719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:18.185159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41108 (local address 127.18.134.3:34549)
0619 02:51:18.185733 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:18.185739 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:18.185769 (+ 30us) server_negotiation.cc:407] Connection header received
0619 02:51:18.185825 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:18.185831 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:18.186044 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:18.186170 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:18.187603 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:18.188655 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:18.189890 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:18.190415 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:18.190512 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:18.194581 (+ 4069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:18.194593 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:18.194609 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:18.194686 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:18.197621 (+ 2935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:18.198288 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:18.198295 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:18.198298 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:18.198469 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:18.198806 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:18.198809 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:18.198811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:18.199186 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:18.199483 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:18.199620 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 02:51:18.199811 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":265,"threads_started":1}
I20260619 02:51:18.201377 27700 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:18.281042 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 27577
I20260619 02:51:18.292677 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 27217
I20260619 02:51:18.293901 27279 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:18.391891 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 27217
I20260619 02:51:18.408584 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:39757
--webserver_interface=127.18.134.62
--webserver_port=33257
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:33407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:39757
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:51:18.716151 27722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:18.716439 27722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:18.716552 27722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:18.723764 27722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:51:18.723904 27722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:18.723980 27722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:18.724066 27722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:51:18.724133 27722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:51:18.733982 27722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:33407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:39757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:39757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=33257
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:18.735285 27722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:18.736546 27722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:18.747694 27727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:18.747660 27730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:18.747727 27728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:18.748524 27722 server_base.cc:1034] running on GCE node
I20260619 02:51:18.749267 27722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:18.750667 27722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:18.751906 27722 hybrid_clock.cc:648] HybridClock initialized: now 1781837478751850 us; error 49 us; skew 500 ppm
I20260619 02:51:18.752264 27722 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:51:18 dist-test-slave-9mr0 krb5kdc[27204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837478, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:51:18.755226 27722 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:51:18.756660 27722 webserver.cc:460] Webserver started at http://127.18.134.62:33257/ using document root <none> and password file <none>
I20260619 02:51:18.757306 27722 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:18.757409 27722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:18.762856 27722 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 02:51:18.765635 27737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:18.767025 27722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:51:18.767169 27722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "4ab8bcdacc8a4709b322cc9a994e4a0e"
format_stamp: "Formatted at 2026-06-19 02:51:16 on dist-test-slave-9mr0"
server_key: "cd9f030c6fe6deb07dcbe085b80e4094"
server_key_iv: "94ab60c0682b9c5219b82c4978871ec9"
server_key_version: "encryptionkey@0"
I20260619 02:51:18.768074 27722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:18.788255 27722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:18.792346 27722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:18.792568 27722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:18.809533 27722 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:39757
I20260619 02:51:18.809551 27788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:39757 every 8 connection(s)
I20260619 02:51:18.811025 27722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:51:18.817730 27789 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:51:18.817740 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 27722
I20260619 02:51:18.818672 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:46743
--local_ip_for_outbound_sockets=127.18.134.1
--tserver_master_addrs=127.18.134.62:39757
--webserver_port=33807
--webserver_interface=127.18.134.1
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:33407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.134.1
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:51:18.822283 27789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e: Bootstrap starting.
I20260619 02:51:18.847662 27789 log.cc:824] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:51:18.858474 27789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:51:18.859150 27789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e: Bootstrap complete.
I20260619 02:51:18.871979 27789 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab8bcdacc8a4709b322cc9a994e4a0e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39757 } }
I20260619 02:51:18.873108 27789 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ab8bcdacc8a4709b322cc9a994e4a0e, State: Initialized, Role: FOLLOWER
I20260619 02:51:18.873813 27789 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab8bcdacc8a4709b322cc9a994e4a0e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39757 } }
I20260619 02:51:18.874075 27789 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:51:18.874194 27789 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:51:18.874364 27789 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:51:18.879798 27789 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab8bcdacc8a4709b322cc9a994e4a0e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39757 } }
I20260619 02:51:18.880355 27789 leader_election.cc:302] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ab8bcdacc8a4709b322cc9a994e4a0e; no voters:
I20260619 02:51:18.881131 27789 leader_election.cc:288] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 02:51:18.881238 27793 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [term 2 FOLLOWER]: Leader election won for term 2
I20260619 02:51:18.882398 27793 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [term 2 LEADER]: Becoming Leader. State: Replica: 4ab8bcdacc8a4709b322cc9a994e4a0e, State: Running, Role: LEADER
I20260619 02:51:18.882948 27793 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab8bcdacc8a4709b322cc9a994e4a0e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39757 } }
I20260619 02:51:18.883345 27789 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:51:18.885223 27795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4ab8bcdacc8a4709b322cc9a994e4a0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab8bcdacc8a4709b322cc9a994e4a0e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39757 } } }
I20260619 02:51:18.885641 27795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [sys.catalog]: This master's current role is: LEADER
I20260619 02:51:18.885589 27794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ab8bcdacc8a4709b322cc9a994e4a0e. Latest consensus state: current_term: 2 leader_uuid: "4ab8bcdacc8a4709b322cc9a994e4a0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ab8bcdacc8a4709b322cc9a994e4a0e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39757 } } }
I20260619 02:51:18.885939 27794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e [sys.catalog]: This master's current role is: LEADER
I20260619 02:51:18.899927 27800 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:51:18.905969 27800 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:51:18.907588 27800 catalog_manager.cc:1259] Loaded cluster ID: 299af98b6cec4579b05626a572a2b38b
I20260619 02:51:18.907784 27800 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:51:18.912117 27800 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:51:18.913756 27800 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4ab8bcdacc8a4709b322cc9a994e4a0e: Loaded TSK: 0
I20260619 02:51:18.915462 27800 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 02:51:19.178956 27791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:19.179227 27791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:19.179342 27791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:19.186834 27791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:19.186944 27791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:19.187099 27791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:51:19.196810 27791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:33407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.1.keytab
--principal=oryx/127.18.134.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:46743
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=33807
--tserver_master_addrs=127.18.134.62:39757
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:19.198048 27791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:19.199350 27791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:19.212215 27816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:19.212371 27815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:19.212568 27818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:19.213078 27791 server_base.cc:1034] running on GCE node
I20260619 02:51:19.213635 27791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:19.214586 27791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:19.215821 27791 hybrid_clock.cc:648] HybridClock initialized: now 1781837479215781 us; error 30 us; skew 500 ppm
I20260619 02:51:19.216120 27791 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:51:19 dist-test-slave-9mr0 krb5kdc[27204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837479, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:51:19.225711 27791 init.cc:377] Logged in from keytab as oryx/127.18.134.1@KRBTEST.COM (short username oryx)
I20260619 02:51:19.227294 27791 webserver.cc:460] Webserver started at http://127.18.134.1:33807/ using document root <none> and password file <none>
I20260619 02:51:19.227950 27791 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:19.228071 27791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:19.233287 27791 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260619 02:51:19.235903 27825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:19.237190 27791 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:51:19.237325 27791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "90bc8507baac465fa1945015c2694378"
format_stamp: "Formatted at 2026-06-19 02:51:16 on dist-test-slave-9mr0"
server_key: "dd2f3ba0d67b0c182e08ead7d6d9c7f7"
server_key_iv: "da05da27e6a36f585367da2eb2a96cb5"
server_key_version: "encryptionkey@0"
I20260619 02:51:19.238206 27791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:19.252316 27791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:19.256278 27791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:19.256510 27791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:19.257484 27791 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:51:19.258857 27791 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:51:19.258957 27791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:19.259058 27791 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:51:19.259142 27791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:19.289037 27791 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:46743
I20260619 02:51:19.289106 27937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:46743 every 8 connection(s)
I20260619 02:51:19.290424 27791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:51:19.295863 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 27791
I20260619 02:51:19.296604 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:39677
--local_ip_for_outbound_sockets=127.18.134.2
--tserver_master_addrs=127.18.134.62:39757
--webserver_port=38375
--webserver_interface=127.18.134.2
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:33407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.134.2
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:51:19 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:19 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
W20260619 02:51:19.603988 27942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:19.604238 27942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:19.604353 27942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:19.611572 27942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:19.611689 27942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:19.611837 27942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:51:19.624410 27942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:33407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.2.keytab
--principal=oryx/127.18.134.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:39677
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=38375
--tserver_master_addrs=127.18.134.62:39757
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:19.625762 27942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:19.627054 27942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:19.642115 27948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:19.642292 27949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:19.642858 27951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:19.643420 27942 server_base.cc:1034] running on GCE node
I20260619 02:51:19.644052 27942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:19.645056 27942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:19.646260 27942 hybrid_clock.cc:648] HybridClock initialized: now 1781837479646193 us; error 60 us; skew 500 ppm
I20260619 02:51:19.646579 27942 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:51:19 dist-test-slave-9mr0 krb5kdc[27204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837479, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:51:19.649583 27942 init.cc:377] Logged in from keytab as oryx/127.18.134.2@KRBTEST.COM (short username oryx)
I20260619 02:51:19.651103 27942 webserver.cc:460] Webserver started at http://127.18.134.2:38375/ using document root <none> and password file <none>
I20260619 02:51:19.651750 27942 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:19.651871 27942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:19.657434 27942 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260619 02:51:19.660396 27958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:19.661751 27942 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 02:51:19.661907 27942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "4d2275694b614a7195cf2d9659ff4217"
format_stamp: "Formatted at 2026-06-19 02:51:17 on dist-test-slave-9mr0"
server_key: "6fc0b9a83ad33fd81db4e211b8a2a4c1"
server_key_iv: "91361ecbc8d5256fca67704f1b73f246"
server_key_version: "encryptionkey@0"
I20260619 02:51:19.662843 27942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:19.680636 27942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:19.684669 27942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:19.684909 27942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:19.685912 27942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:51:19.687307 27942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:51:19.687387 27942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:19.687489 27942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:51:19.687551 27942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:19.725414 27942 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:39677
I20260619 02:51:19.725471 28070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:39677 every 8 connection(s)
I20260619 02:51:19.726897 27942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:51:19.727545 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 27942
I20260619 02:51:19.728327 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:34549
--local_ip_for_outbound_sockets=127.18.134.3
--tserver_master_addrs=127.18.134.62:39757
--webserver_port=40427
--webserver_interface=127.18.134.3
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:33407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.134.3
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:51:19 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:19 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
W20260619 02:51:20.054155 28075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:20.054448 28075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:20.054610 28075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:20.061986 28075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:20.062109 28075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:20.062288 28075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:51:20.072212 28075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:33407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/krb5kdc/oryx_127.18.134.3.keytab
--principal=oryx/127.18.134.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:34549
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=40427
--tserver_master_addrs=127.18.134.62:39757
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:20.073556 28075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:20.074827 28075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:20.089838 28081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:20.094234 28082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:20.094756 28084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:20.095548 28075 server_base.cc:1034] running on GCE node
I20260619 02:51:20.096131 28075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:20.097100 28075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:20.098322 28075 hybrid_clock.cc:648] HybridClock initialized: now 1781837480098258 us; error 43 us; skew 500 ppm
I20260619 02:51:20.098670 28075 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:51:20 dist-test-slave-9mr0 krb5kdc[27204](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837480, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.134.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:51:20.101907 28075 init.cc:377] Logged in from keytab as oryx/127.18.134.3@KRBTEST.COM (short username oryx)
I20260619 02:51:20.108179 28075 webserver.cc:460] Webserver started at http://127.18.134.3:40427/ using document root <none> and password file <none>
I20260619 02:51:20.108877 28075 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:20.109010 28075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:20.114307 28075 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.001s
I20260619 02:51:20.116870 28091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:20.118218 28075 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 02:51:20.118367 28075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "36b8df26080949bba15dd77661caf470"
format_stamp: "Formatted at 2026-06-19 02:51:17 on dist-test-slave-9mr0"
server_key: "5ad2538f67f597c549cd3e434993a40c"
server_key_iv: "fb3ae2bc6e4de28acfb02e9b99de04d6"
server_key_version: "encryptionkey@0"
I20260619 02:51:20.119319 28075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:20.143168 28075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:20.147269 28075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:20.147511 28075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:20.148505 28075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:51:20.149844 28075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:51:20.149940 28075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:20.150048 28075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:51:20.150116 28075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:20.187806 28075 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:34549
I20260619 02:51:20.187852 28203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:34549 every 8 connection(s)
I20260619 02:51:20.189205 28075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:51:20.193675 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 28075
Jun 19 02:51:20 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:20 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:20.225680 28207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:20.195473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47274 (local address 127.18.134.62:39757)
0619 02:51:20.204126 (+ 8653us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:20.204132 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:20.204154 (+ 22us) server_negotiation.cc:407] Connection header received
0619 02:51:20.204222 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:20.204229 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:20.204312 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:20.204439 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:20.208036 (+ 3597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:20.209202 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:20.211014 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:20.211592 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:20.211715 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:20.215257 (+ 3542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:20.215269 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:20.215289 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:20.215338 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:20.219553 (+ 4215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:20.220839 (+ 1286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:20.220850 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:20.220860 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:20.221079 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:20.221942 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:20.221948 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:20.221951 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:20.222729 (+ 778us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:20.223959 (+ 1230us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:20.224233 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 02:51:20.224690 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8522,"thread_start_us":7381,"threads_started":1}
W20260619 02:51:20.426895 27941 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.123s user 0.004s sys 0.007s
I20260619 02:51:20.428680 27941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:19.293251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.1:41941)
0619 02:51:19.294014 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:19.294073 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:19.295863 (+ 1790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:19.297431 (+ 1568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:19.297438 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:19.297919 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:19.298437 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:19.298491 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:19.300506 (+ 2015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:19.300513 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:19.301976 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:19.301995 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:19.303254 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:19.303262 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:19.303552 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:19.304251 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:19.304273 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:20.427839 (+1123566us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":339,"threads_started":1}
W20260619 02:51:20.429596 27938 heartbeater.cc:644] Failed to heartbeat to 127.18.134.62:39757 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.134.62:39757: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
W20260619 02:51:20.430160 27940 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:19.293203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:41941 (local address 127.18.134.62:39757)
0619 02:51:19.293801 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:19.293813 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:19.294181 (+ 368us) server_negotiation.cc:407] Connection header received
0619 02:51:19.296459 (+ 2278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:19.296477 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:19.296922 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:19.297306 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:19.298673 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:19.300327 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:19.302145 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:19.303018 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:19.303158 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:20.429879 (+1126721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.1:41941: BlockingRecv error: recv got EOF from 127.18.134.1:41941 (error 108)
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":232,"threads_started":1}
Jun 19 02:51:20 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:20 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:20.440647 27941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:20.430882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.1:47209)
0619 02:51:20.431488 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:20.431509 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:20.431617 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:20.432716 (+ 1099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:20.432721 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:20.433186 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:20.433390 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:20.433403 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:20.434456 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:20.434460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:20.435449 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:20.435460 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:20.436367 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:20.436374 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:20.436521 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:20.437081 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:20.437097 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:20.440347 (+ 3250us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":768,"threads_started":1}
W20260619 02:51:20.440778 27940 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:20.432075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:47209 (local address 127.18.134.62:39757)
0619 02:51:20.432244 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:20.432250 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:20.432264 (+ 14us) server_negotiation.cc:407] Connection header received
0619 02:51:20.432307 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:20.432311 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:20.432386 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:20.432477 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:20.433553 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:20.434304 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:20.435663 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:20.436201 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:20.436315 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:20.440560 (+ 4245us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.1:47209: BlockingRecv error: recv got EOF from 127.18.134.1:47209 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 19 02:51:20 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:20 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:20.450343 27941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:20.441622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.1:43835)
0619 02:51:20.441906 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:20.441919 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:20.442030 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:20.442347 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:20.442352 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:20.442751 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:20.442924 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:20.442933 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:20.443959 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:20.443965 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:20.444990 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:20.445001 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:20.445849 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:20.445855 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:20.446005 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:20.447060 (+ 1055us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:20.447082 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:20.450083 (+ 3001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208}
W20260619 02:51:20.450500 27940 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:20.441675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:43835 (local address 127.18.134.62:39757)
0619 02:51:20.441816 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:20.441820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:20.442008 (+ 188us) server_negotiation.cc:407] Connection header received
0619 02:51:20.442124 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:20.442129 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:20.442230 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:20.442310 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:20.443047 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:20.443799 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:20.445117 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:20.445703 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:20.445816 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:20.450265 (+ 4449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.1:43835: BlockingRecv error: recv got EOF from 127.18.134.1:43835 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260619 02:51:20.450906 27938 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:51:20.958984 28074 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.205s user 0.001s sys 0.018s
W20260619 02:51:20.960376 28073 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:19.730118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:47181 (local address 127.18.134.62:39757)
0619 02:51:19.743068 (+ 12950us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:19.743073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:19.743092 (+ 19us) server_negotiation.cc:407] Connection header received
0619 02:51:19.745498 (+ 2406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:19.745504 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:19.745582 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:19.745701 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:19.748021 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:19.749139 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:19.751692 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:19.752316 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:19.752452 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:20.960178 (+1207726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.2:47181: BlockingRecv error: recv got EOF from 127.18.134.2:47181 (error 108)
Metrics: {"server-negotiator.queue_time_us":12844,"thread_start_us":12740,"threads_started":1}
I20260619 02:51:20.961396 28074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:19.730978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.2:47181)
0619 02:51:19.742279 (+ 11301us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:19.742355 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:19.745136 (+ 2781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:19.746015 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:19.746031 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:19.746888 (+ 857us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:19.747797 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:19.747824 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:19.749309 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:19.749317 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:19.751525 (+ 2208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:19.751542 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:19.752583 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:19.752590 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:19.752912 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:19.753741 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:19.753779 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:20.960069 (+1206290us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":10896,"thread_start_us":10639,"threads_started":1}
W20260619 02:51:20.962349 28071 heartbeater.cc:644] Failed to heartbeat to 127.18.134.62:39757 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.134.62:39757: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Jun 19 02:51:20 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:20 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:20.972460 28074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:20.963001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.2:57303)
0619 02:51:20.963192 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:20.963212 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:20.963307 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:20.963627 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:20.963630 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:20.963938 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:20.964097 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:20.964105 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:20.965622 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:20.965625 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:20.966748 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:20.966761 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:20.967768 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:20.967774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:20.967938 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:20.968669 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:20.968693 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:20.972225 (+ 3532us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":72}
W20260619 02:51:20.972796 28073 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:20.963108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:57303 (local address 127.18.134.62:39757)
0619 02:51:20.963305 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:20.963312 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:20.963336 (+ 24us) server_negotiation.cc:407] Connection header received
0619 02:51:20.963403 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:20.963409 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:20.963494 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:20.963618 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:20.964221 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:20.965495 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:20.966884 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:20.967594 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:20.967729 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:20.972529 (+ 4800us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.2:57303: BlockingRecv error: recv got EOF from 127.18.134.2:57303 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Jun 19 02:51:20 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:20 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:20.982405 28074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:20.973120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.2:59321)
0619 02:51:20.973261 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:20.973278 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:20.973375 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:20.974079 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:20.974082 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:20.974347 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:20.974494 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:20.974502 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:20.976062 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:20.976066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:20.976984 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:20.976995 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:20.977903 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:20.977907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:20.978033 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:20.978652 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:20.978675 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:20.982209 (+ 3534us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":44}
W20260619 02:51:20.982833 28071 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:51:20.982844 28073 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:20.973576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:59321 (local address 127.18.134.62:39757)
0619 02:51:20.973738 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:20.973744 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:20.973766 (+ 22us) server_negotiation.cc:407] Connection header received
0619 02:51:20.973824 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:20.973831 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:20.973910 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:20.974026 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:20.974687 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:20.975919 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:20.977106 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:20.977775 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:20.977905 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:20.982515 (+ 4610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.2:59321: BlockingRecv error: recv got EOF from 127.18.134.2:59321 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260619 02:51:21.296434 28206 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.077s user 0.000s sys 0.016s
W20260619 02:51:21.297734 28209 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:20.194426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:46839 (local address 127.18.134.62:39757)
0619 02:51:20.203085 (+ 8659us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:20.203092 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:20.207862 (+ 4770us) server_negotiation.cc:407] Connection header received
0619 02:51:20.210554 (+ 2692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:20.210560 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:20.210661 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:20.210757 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:20.213178 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:20.214391 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:20.216812 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:20.217484 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:20.217582 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:21.297444 (+1079862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.3:46839: BlockingRecv error: recv got EOF from 127.18.134.3:46839 (error 108)
Metrics: {"server-negotiator.queue_time_us":8531,"thread_start_us":9455,"threads_started":1}
I20260619 02:51:21.298259 28206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:20.192765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.3:46839)
0619 02:51:20.207319 (+ 14554us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:20.207381 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:20.210105 (+ 2724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:20.211249 (+ 1144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:20.211263 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:20.212095 (+ 832us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:20.212970 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:20.213001 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:20.214598 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:20.214602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:20.216648 (+ 2046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:20.216664 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:20.217959 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:20.217966 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:20.218267 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:20.218942 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:20.218972 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:21.297231 (+1078259us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":14221,"thread_start_us":13986,"threads_started":1}
W20260619 02:51:21.299046 28204 heartbeater.cc:644] Failed to heartbeat to 127.18.134.62:39757 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.134.62:39757: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Jun 19 02:51:21 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:21 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:21.307626 28206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:21.299657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.3:45383)
0619 02:51:21.299860 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:21.299894 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:21.299991 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:21.300323 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:21.300327 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:21.300683 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:21.300870 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:21.300881 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:21.301920 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:21.301925 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:21.302936 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:21.302949 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:21.303606 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:21.303610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:21.303722 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:21.304226 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:21.304242 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:21.307425 (+ 3183us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":69}
W20260619 02:51:21.307893 28209 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:21.299814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:45383 (local address 127.18.134.62:39757)
0619 02:51:21.299954 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:21.299958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:21.299973 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:51:21.300111 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:21.300116 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:21.300172 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:21.300247 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:21.301005 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:21.301767 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:21.303075 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:21.303462 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:21.303554 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:21.307637 (+ 4083us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.3:45383: BlockingRecv error: recv got EOF from 127.18.134.3:45383 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Jun 19 02:51:21 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:21 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:21.316154 28206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:21.308190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.3:44237)
0619 02:51:21.308394 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:21.308411 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:21.308512 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:21.308865 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:21.308869 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:21.309171 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:21.309338 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:21.309346 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:21.310430 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:21.310434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:21.311433 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:21.311444 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:21.312247 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:21.312251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:21.312431 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:21.312932 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:21.312949 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:21.315968 (+ 3019us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113}
W20260619 02:51:21.316414 28209 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:21.308312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:44237 (local address 127.18.134.62:39757)
0619 02:51:21.308558 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:21.308564 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:21.308581 (+ 17us) server_negotiation.cc:407] Connection header received
0619 02:51:21.308628 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:21.308632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:21.308704 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:21.308803 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:21.309460 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:21.310300 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:21.311634 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:21.312085 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:21.312183 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:21.316161 (+ 3978us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.3:44237: BlockingRecv error: recv got EOF from 127.18.134.3:44237 (error 108)
Metrics: {"server-negotiator.queue_time_us":144}
W20260619 02:51:21.316428 28204 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 19 02:51:21 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:21 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:21.460430 28212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:21.451498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.1:42931)
0619 02:51:21.451993 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:21.452021 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:21.452167 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:21.452591 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:21.452596 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:21.453105 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:21.453284 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:21.453296 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:21.454299 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:21.454303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:21.455323 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:21.455335 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:21.456083 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:21.456087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:21.456221 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:21.456759 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:21.456776 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:21.460123 (+ 3347us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":277,"threads_started":1}
W20260619 02:51:21.460541 28209 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:21.451668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:42931 (local address 127.18.134.62:39757)
0619 02:51:21.451871 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:21.451877 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:21.452135 (+ 258us) server_negotiation.cc:407] Connection header received
0619 02:51:21.452294 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:21.452300 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:21.452441 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:21.452533 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:21.453445 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:21.454143 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:21.455469 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:21.455925 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:21.456028 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:21.460337 (+ 4309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.1:42931: BlockingRecv error: recv got EOF from 127.18.134.1:42931 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Jun 19 02:51:21 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:21 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:21.992153 28213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:21.983555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.2:48663)
0619 02:51:21.984110 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:21.984135 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:21.984247 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:21.984769 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:21.984774 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:21.985216 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:21.985356 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:21.985365 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:21.986430 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:21.986434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:21.987422 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:21.987435 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:21.988089 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:21.988093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:21.988203 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:21.988745 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:21.988763 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:21.991906 (+ 3143us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":275,"threads_started":1}
W20260619 02:51:21.992322 28214 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:21.983577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:48663 (local address 127.18.134.62:39757)
0619 02:51:21.984472 (+ 895us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:21.984477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:21.984493 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:51:21.984539 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:21.984544 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:21.984611 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:21.984709 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:21.985517 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:21.986275 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:21.987543 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:21.987955 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:21.988047 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:21.992098 (+ 4051us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.2:48663: BlockingRecv error: recv got EOF from 127.18.134.2:48663 (error 108)
Metrics: {"server-negotiator.queue_time_us":794,"thread_start_us":464,"threads_started":1}
Jun 19 02:51:22 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:22 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:22.325563 28215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:22.317014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.3:36165)
0619 02:51:22.317551 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:22.317567 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:22.317680 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:22.317934 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:22.317937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:22.318314 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:22.318452 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:22.318459 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:22.319481 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:22.319485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:22.320425 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:22.320437 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:22.321048 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:22.321052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:22.321163 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:22.321706 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:22.321722 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:22.325244 (+ 3522us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":292,"threads_started":1}
W20260619 02:51:22.325657 28214 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:22.317097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:36165 (local address 127.18.134.62:39757)
0619 02:51:22.317244 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:22.317249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:22.317632 (+ 383us) server_negotiation.cc:407] Connection header received
0619 02:51:22.317765 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:22.317768 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:22.317827 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:22.317905 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:22.318582 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:22.319351 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:22.320536 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:22.320917 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:22.321013 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:22.325444 (+ 4431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.3:36165: BlockingRecv error: recv got EOF from 127.18.134.3:36165 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Jun 19 02:51:22 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:22 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:22.469774 28216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:22.461328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.1:33217)
0619 02:51:22.461784 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:22.461802 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:22.461931 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:22.462234 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:22.462238 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:22.462694 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:22.462875 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:22.462884 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:22.463874 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:22.463878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:22.464778 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:22.464790 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:22.465467 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:22.465473 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:22.465627 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:22.466237 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:22.466256 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:22.469419 (+ 3163us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":261,"threads_started":1}
W20260619 02:51:22.469841 28214 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:22.461408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:33217 (local address 127.18.134.62:39757)
0619 02:51:22.461607 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:22.461613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:22.461875 (+ 262us) server_negotiation.cc:407] Connection header received
0619 02:51:22.462040 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:22.462046 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:22.462112 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:22.462190 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:22.463010 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:22.463741 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:22.464904 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:22.465312 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:22.465417 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:22.469593 (+ 4176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.1:33217: BlockingRecv error: recv got EOF from 127.18.134.1:33217 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 19 02:51:22 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:22 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:23.002144 28217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:22.993097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.2:52627)
0619 02:51:22.993642 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:22.993664 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:22.993777 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:22.994226 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:22.994229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:22.994673 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:22.994865 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:22.994878 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:22.995867 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:22.995871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:22.996783 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:22.996793 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:22.997680 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:22.997686 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:22.997857 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:22.998440 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:22.998458 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:23.001817 (+ 3359us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":264,"threads_started":1}
W20260619 02:51:23.002270 28218 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:22.993197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:52627 (local address 127.18.134.62:39757)
0619 02:51:22.993925 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:22.993929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:22.993962 (+ 33us) server_negotiation.cc:407] Connection header received
0619 02:51:22.994017 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:22.994021 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:22.994076 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:22.994162 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:22.995033 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:22.995732 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:22.997000 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:22.997529 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:22.997655 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:23.002038 (+ 4383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.2:52627: BlockingRecv error: recv got EOF from 127.18.134.2:52627 (error 108)
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":544,"threads_started":1}
Jun 19 02:51:23 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:23 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:23.335089 28219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:23.326495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.3:34925)
0619 02:51:23.327022 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:23.327045 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:23.327158 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:23.327458 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:23.327462 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:23.327863 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:23.328053 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:23.328062 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:23.329014 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:23.329019 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:23.329975 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:23.329986 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:23.330598 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:23.330603 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:23.330778 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:23.331335 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:23.331352 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:23.334764 (+ 3412us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":301,"threads_started":1}
W20260619 02:51:23.335194 28218 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:23.326733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:34925 (local address 127.18.134.62:39757)
0619 02:51:23.326880 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:23.326885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:23.327111 (+ 226us) server_negotiation.cc:407] Connection header received
0619 02:51:23.327240 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:23.327244 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:23.327300 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:23.327378 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:23.328170 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:23.328892 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:23.330091 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:23.330468 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:23.330561 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:23.334958 (+ 4397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.3:34925: BlockingRecv error: recv got EOF from 127.18.134.3:34925 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Jun 19 02:51:23 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:23 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:23.479768 28220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:23.470905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.1:45433)
0619 02:51:23.471449 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:23.471471 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:23.471651 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:23.472000 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:23.472005 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:23.472424 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:23.472582 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:23.472591 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:23.473676 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:23.473680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:23.474584 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:23.474595 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:23.475323 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:23.475328 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:23.475476 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:23.476026 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:23.476043 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:23.479454 (+ 3411us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":268,"threads_started":1}
W20260619 02:51:23.479924 28218 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:23.471022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:45433 (local address 127.18.134.62:39757)
0619 02:51:23.471200 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:23.471206 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:23.471560 (+ 354us) server_negotiation.cc:407] Connection header received
0619 02:51:23.471770 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:23.471776 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:23.471864 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:23.471947 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:23.472732 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:23.473550 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:23.474753 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:23.475187 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:23.475290 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:23.479668 (+ 4378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.1:45433: BlockingRecv error: recv got EOF from 127.18.134.1:45433 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Jun 19 02:51:24 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:24 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:24.013145 28221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:24.003177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.2:41149)
0619 02:51:24.003679 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:24.003704 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:24.003838 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:24.004185 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:24.004189 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:24.004628 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:24.004794 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:24.004802 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:24.005980 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:24.005987 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:24.007431 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:24.007446 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:24.008379 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:24.008385 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:24.008557 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:24.009146 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:24.009164 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:24.012790 (+ 3626us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":223,"threads_started":1}
W20260619 02:51:24.013887 28222 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:24.003489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:41149 (local address 127.18.134.62:39757)
0619 02:51:24.003879 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:24.003885 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:24.003903 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:51:24.003956 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:24.003961 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:24.004050 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:24.004155 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:24.004948 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:24.005838 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:24.007578 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:24.008239 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:24.008377 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:24.013610 (+ 5233us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.2:41149: BlockingRecv error: recv got EOF from 127.18.134.2:41149 (error 108)
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":199,"threads_started":1}
Jun 19 02:51:24 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:24 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:24.362704 28223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:24.336098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.3:59469)
0619 02:51:24.336608 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:24.336628 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:24.336746 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:24.337046 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:24.337050 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:24.337406 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:24.337572 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:24.337582 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:24.338589 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:24.338592 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:24.339557 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:24.339570 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:24.340318 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:24.340322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:24.340449 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:24.359112 (+ 18663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:24.359136 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:24.362383 (+ 3247us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":263,"threads_started":1}
W20260619 02:51:24.362844 28222 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:24.336239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:59469 (local address 127.18.134.62:39757)
0619 02:51:24.336410 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:24.336414 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:24.336739 (+ 325us) server_negotiation.cc:407] Connection header received
0619 02:51:24.336847 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:24.336851 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:24.336915 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:24.336999 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:24.337728 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:24.338469 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:24.339704 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:24.340169 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:24.340285 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:24.362602 (+ 22317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.3:59469: BlockingRecv error: recv got EOF from 127.18.134.3:59469 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Jun 19 02:51:24 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:24 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:24.489619 28224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:24.480831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.1:35991)
0619 02:51:24.481331 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:24.481350 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:24.481501 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:24.481833 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:24.481837 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:24.482231 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:24.482395 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:24.482404 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:24.483464 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:24.483469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:24.484432 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:24.484470 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:24.485139 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:24.485145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:24.485318 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:24.485878 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:24.485897 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:24.489312 (+ 3415us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":277,"threads_started":1}
W20260619 02:51:24.489681 28222 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:24.480889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:35991 (local address 127.18.134.62:39757)
0619 02:51:24.481166 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:24.481173 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:24.481443 (+ 270us) server_negotiation.cc:407] Connection header received
0619 02:51:24.481629 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:24.481634 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:24.481704 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:24.481786 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:24.482532 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:24.483343 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:24.484579 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:24.484989 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:24.485127 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:24.489487 (+ 4360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.1:35991: BlockingRecv error: recv got EOF from 127.18.134.1:35991 (error 108)
Metrics: {"server-negotiator.queue_time_us":141}
Jun 19 02:51:25 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:25 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:25.023042 28225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:25.014146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.2:36569)
0619 02:51:25.014702 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:25.014724 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:25.014832 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:25.015199 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:25.015202 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:25.015618 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:25.015784 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:25.015793 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:25.016961 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:25.016966 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:25.017927 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:25.017939 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:25.018616 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:25.018619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:25.018764 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:25.019359 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:25.019376 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:25.022794 (+ 3418us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":263,"threads_started":1}
W20260619 02:51:25.023208 28226 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:25.014242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:36569 (local address 127.18.134.62:39757)
0619 02:51:25.014770 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:25.014775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:25.014797 (+ 22us) server_negotiation.cc:407] Connection header received
0619 02:51:25.014949 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:25.014953 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:25.015046 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:25.015170 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:25.015938 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:25.016801 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:25.018058 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:25.018500 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:25.018599 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:25.022987 (+ 4388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.2:36569: BlockingRecv error: recv got EOF from 127.18.134.2:36569 (error 108)
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":299,"threads_started":1}
Jun 19 02:51:25 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:25 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:25.372784 28227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:25.363656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.3:51391)
0619 02:51:25.364198 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:25.364214 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:25.364328 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:25.364707 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:25.364712 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:25.365071 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:25.365229 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:25.365238 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:25.366360 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:25.366364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:25.367491 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:25.367505 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:25.368208 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:25.368213 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:25.368348 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:25.368955 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:25.368972 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:25.372544 (+ 3572us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":293,"threads_started":1}
W20260619 02:51:25.372918 28226 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:25.363785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:51391 (local address 127.18.134.62:39757)
0619 02:51:25.363944 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:25.363949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:25.364317 (+ 368us) server_negotiation.cc:407] Connection header received
0619 02:51:25.364472 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:25.364476 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:25.364539 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:25.364625 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:25.365388 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:25.366209 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:25.367635 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:25.368051 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:25.368153 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:25.372719 (+ 4566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.3:51391: BlockingRecv error: recv got EOF from 127.18.134.3:51391 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Jun 19 02:51:25 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:25 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:25.498972 28228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:25.490510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.1:44013)
0619 02:51:25.491015 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:25.491034 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:25.491169 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:25.491512 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:25.491516 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:25.491928 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:25.492107 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:25.492118 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:25.493231 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:25.493235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:25.494162 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:25.494175 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:25.494871 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:25.494875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:25.495004 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:25.495563 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:25.495579 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:25.498746 (+ 3167us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":271,"threads_started":1}
W20260619 02:51:25.499205 28226 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:25.490624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:44013 (local address 127.18.134.62:39757)
0619 02:51:25.490848 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:25.490854 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:25.491118 (+ 264us) server_negotiation.cc:407] Connection header received
0619 02:51:25.491280 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:25.491284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:25.491364 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:25.491465 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:25.492237 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:25.493081 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:25.494298 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:25.494752 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:25.494864 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:25.498959 (+ 4095us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.1:44013: BlockingRecv error: recv got EOF from 127.18.134.1:44013 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Jun 19 02:51:26 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:26 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:26.032902 28229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:26.023993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.2:38253)
0619 02:51:26.024537 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:26.024559 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:26.024663 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:26.024994 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:26.025000 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:26.025427 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:26.025568 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:26.025577 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:26.026601 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:26.026604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:26.027562 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:26.027573 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:26.028321 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:26.028326 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:26.028476 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:26.029047 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:26.029066 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:26.032597 (+ 3531us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":253,"threads_started":1}
W20260619 02:51:26.033068 28230 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:26.024190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:38253 (local address 127.18.134.62:39757)
0619 02:51:26.024601 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:26.024606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:26.024624 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:51:26.024759 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:26.024763 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:26.024829 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:26.024943 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:26.025714 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:26.026489 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:26.027693 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:26.028116 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:26.028217 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:26.032822 (+ 4605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.2:38253: BlockingRecv error: recv got EOF from 127.18.134.2:38253 (error 108)
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":224,"threads_started":1}
Jun 19 02:51:26 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:26 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:26.382763 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:26.373723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.3:60479)
0619 02:51:26.374226 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:26.374242 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:26.374372 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:26.374762 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:26.374767 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:26.375258 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:26.375482 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:26.375496 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:26.376741 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:26.376745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:26.377653 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:26.377665 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:26.378318 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:26.378322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:26.378517 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:26.379107 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:26.379124 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:26.382414 (+ 3290us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":269,"threads_started":1}
W20260619 02:51:26.382949 28230 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:26.373824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:60479 (local address 127.18.134.62:39757)
0619 02:51:26.373997 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:26.374002 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:26.374337 (+ 335us) server_negotiation.cc:407] Connection header received
0619 02:51:26.374466 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:26.374471 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:26.374544 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:26.374666 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:26.375635 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:26.376611 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:26.377783 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:26.378182 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:26.378270 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:26.382699 (+ 4429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.3:60479: BlockingRecv error: recv got EOF from 127.18.134.3:60479 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Jun 19 02:51:26 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:26 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:26.508173 28232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:26.499863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.1:34427)
0619 02:51:26.500300 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:26.500321 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:26.500429 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:26.500762 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:26.500767 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:26.501170 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:26.501336 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:26.501344 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:26.502380 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:26.502384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:26.503332 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:26.503343 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:26.504044 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:26.504049 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:26.504166 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:26.504720 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:26.504737 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:26.507941 (+ 3204us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":226,"threads_started":1}
W20260619 02:51:26.508433 28230 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:26.499926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:34427 (local address 127.18.134.62:39757)
0619 02:51:26.500074 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:26.500079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:26.500388 (+ 309us) server_negotiation.cc:407] Connection header received
0619 02:51:26.500525 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:26.500529 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:26.500604 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:26.500687 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:26.501483 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:26.502246 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:26.503467 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:26.503887 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:26.503991 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:26.508164 (+ 4173us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.1:34427: BlockingRecv error: recv got EOF from 127.18.134.1:34427 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Jun 19 02:51:27 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:27 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:27.042847 28233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:27.033871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.2:53843)
0619 02:51:27.034338 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:27.034360 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:27.034470 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:27.034824 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:27.034827 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:27.035275 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:27.035467 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:27.035481 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:27.036763 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:27.036767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:27.037665 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:27.037676 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:27.038462 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:27.038468 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:27.038601 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:27.039197 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:27.039215 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:27.042573 (+ 3358us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":229,"threads_started":1}
W20260619 02:51:27.043135 28234 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:27.033979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:53843 (local address 127.18.134.62:39757)
0619 02:51:27.034398 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:27.034403 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:27.034421 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:51:27.034575 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:27.034579 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:27.034676 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:27.034778 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:27.035628 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:27.036625 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:27.037782 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:27.038211 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:27.038303 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:27.042865 (+ 4562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.2:53843: BlockingRecv error: recv got EOF from 127.18.134.2:53843 (error 108)
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":234,"threads_started":1}
Jun 19 02:51:27 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:27 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:27.392836 28235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:27.384039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.3:33095)
0619 02:51:27.384533 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:27.384549 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:27.384668 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:27.384950 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:27.384954 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:27.385354 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:27.385510 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:27.385521 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:27.386621 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:27.386668 (+ 47us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:27.387785 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:27.387801 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:27.388464 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:27.388468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:27.388590 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:27.389144 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:27.389161 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:27.392603 (+ 3442us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":256,"threads_started":1}
W20260619 02:51:27.392995 28234 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:27.384205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:33095 (local address 127.18.134.62:39757)
0619 02:51:27.384441 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:27.384448 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:27.384621 (+ 173us) server_negotiation.cc:407] Connection header received
0619 02:51:27.384758 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:27.384762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:27.384826 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:27.384908 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:27.385676 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:27.386434 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:27.387933 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:27.388317 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:27.388409 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:27.392809 (+ 4400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.3:33095: BlockingRecv error: recv got EOF from 127.18.134.3:33095 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Jun 19 02:51:27 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:27 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:27.517405 28236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:27.509084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.1:54091)
0619 02:51:27.509640 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:27.509659 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:27.509775 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:27.510048 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:27.510051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:27.510414 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:27.510598 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:27.510606 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:27.511659 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:27.511663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:27.512594 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:27.512606 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:27.513296 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:27.513300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:27.513404 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:27.513977 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:27.513994 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:27.517150 (+ 3156us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":266,"threads_started":1}
W20260619 02:51:27.517596 28234 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:27.509168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:54091 (local address 127.18.134.62:39757)
0619 02:51:27.509351 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:27.509358 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:27.509733 (+ 375us) server_negotiation.cc:407] Connection header received
0619 02:51:27.509861 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:27.509865 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:27.509939 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:27.510014 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:27.510750 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:27.511517 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:27.512738 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:27.513167 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:27.513267 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:27.517339 (+ 4072us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.1:54091: BlockingRecv error: recv got EOF from 127.18.134.1:54091 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 19 02:51:28 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:28 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:28.052971 28238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:28.043747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.2:41371)
0619 02:51:28.044787 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:28.044811 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:28.044942 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:28.045283 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:28.045288 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:28.045688 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:28.045849 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:28.045859 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:28.046857 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:28.046862 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:28.047802 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:28.047814 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:28.048501 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:28.048507 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:28.048669 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:28.049218 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:28.049235 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:28.052723 (+ 3488us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":935,"thread_start_us":706,"threads_started":1}
W20260619 02:51:28.053169 28237 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:28.043845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:41371 (local address 127.18.134.62:39757)
0619 02:51:28.044585 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:28.044591 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:28.044899 (+ 308us) server_negotiation.cc:407] Connection header received
0619 02:51:28.045053 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:28.045059 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:28.045134 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:28.045219 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:28.045989 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:28.046730 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:28.047941 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:28.048357 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:28.048467 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:28.052913 (+ 4446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.2:41371: BlockingRecv error: recv got EOF from 127.18.134.2:41371 (error 108)
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":514,"threads_started":1}
Jun 19 02:51:28 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:28 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:28.402405 28239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:28.393890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.3:35039)
0619 02:51:28.394508 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:28.394523 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:28.394659 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:28.394991 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:28.394995 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:28.395382 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:28.395543 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:28.395551 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:28.396536 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:28.396541 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:28.397457 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:28.397467 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:28.398112 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:28.398118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:28.398266 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:28.398854 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:28.398871 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:28.402065 (+ 3194us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":241,"threads_started":1}
W20260619 02:51:28.402522 28237 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:28.393959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:35039 (local address 127.18.134.62:39757)
0619 02:51:28.394123 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:28.394130 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:28.394597 (+ 467us) server_negotiation.cc:407] Connection header received
0619 02:51:28.394781 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:28.394786 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:28.394863 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:28.394944 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:28.395674 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:28.396407 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:28.397583 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:28.397979 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:28.398081 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:28.402283 (+ 4202us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.3:35039: BlockingRecv error: recv got EOF from 127.18.134.3:35039 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 19 02:51:28 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:28 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:28.526912 28240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:28.518288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.1:57057)
0619 02:51:28.518881 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:28.518900 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:28.519059 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:28.519358 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:28.519362 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:28.519747 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:28.519904 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:28.519913 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:28.520954 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:28.520958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:28.521886 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:28.521898 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:28.522689 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:28.522696 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:28.522841 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:28.523439 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:28.523458 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:28.526571 (+ 3113us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":327,"threads_started":1}
W20260619 02:51:28.527092 28237 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:28.518521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:57057 (local address 127.18.134.62:39757)
0619 02:51:28.518781 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:28.518788 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:28.519004 (+ 216us) server_negotiation.cc:407] Connection header received
0619 02:51:28.519168 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:28.519173 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:28.519236 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:28.519313 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:28.520048 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:28.520807 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:28.522026 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:28.522492 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:28.522599 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:28.526828 (+ 4229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.1:57057: BlockingRecv error: recv got EOF from 127.18.134.1:57057 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Jun 19 02:51:29 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:29 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:29.064481 28241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:29.053824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.2:38887)
0619 02:51:29.054336 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:29.054359 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:29.054476 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:29.054774 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:29.054777 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:29.055125 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:29.055276 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:29.055284 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:29.057062 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:29.057068 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:29.058492 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:29.058508 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:29.059470 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:29.059477 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:29.059653 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:29.060259 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:29.060282 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:29.064157 (+ 3875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":250,"threads_started":1}
W20260619 02:51:29.065177 28242 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:29.053930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:38887 (local address 127.18.134.62:39757)
0619 02:51:29.054418 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:29.054424 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:29.054440 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:51:29.054566 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:29.054569 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:29.054653 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:29.054736 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:29.055490 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:29.056908 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:29.058659 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:29.059280 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:29.059395 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:29.064927 (+ 5532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.2:38887: BlockingRecv error: recv got EOF from 127.18.134.2:38887 (error 108)
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":221,"threads_started":1}
Jun 19 02:51:29 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:29 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:29.415644 28243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:29.403365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.3:41243)
0619 02:51:29.403903 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:29.403920 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:29.404038 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:29.404359 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:29.404363 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:29.404753 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:29.404915 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:29.404925 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:29.406067 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:29.406071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:29.407081 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:29.407095 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:29.407916 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:29.407922 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:29.408074 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:29.412075 (+ 4001us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:29.412098 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:29.415384 (+ 3286us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":286,"threads_started":1}
W20260619 02:51:29.415817 28242 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:29.403476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:41243 (local address 127.18.134.62:39757)
0619 02:51:29.403654 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:29.403659 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:29.404025 (+ 366us) server_negotiation.cc:407] Connection header received
0619 02:51:29.404131 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:29.404135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:29.404197 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:29.404284 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:29.405078 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:29.405875 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:29.407207 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:29.407718 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:29.407856 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:29.415592 (+ 7736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.3:41243: BlockingRecv error: recv got EOF from 127.18.134.3:41243 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Jun 19 02:51:29 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:29 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:29.536902 28244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:29.528048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.1:49041)
0619 02:51:29.528543 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:29.528562 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:29.528701 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:29.529053 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:29.529058 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:29.529509 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:29.529680 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:29.529691 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:29.530769 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:29.530775 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:29.531695 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:29.531706 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:29.532384 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:29.532389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:29.532565 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:29.533129 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:29.533146 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:29.536643 (+ 3497us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":274,"threads_started":1}
W20260619 02:51:29.537086 28242 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:29.528168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:49041 (local address 127.18.134.62:39757)
0619 02:51:29.528386 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:29.528393 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:29.528644 (+ 251us) server_negotiation.cc:407] Connection header received
0619 02:51:29.528826 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:29.528830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:29.528915 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:29.528997 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:29.529827 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:29.530615 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:29.531832 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:29.532242 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:29.532348 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:29.536863 (+ 4515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.1:49041: BlockingRecv error: recv got EOF from 127.18.134.1:49041 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Jun 19 02:51:30 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:30 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:30.074904 28245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:30.065575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.2:53721)
0619 02:51:30.066033 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:30.066054 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:30.066165 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:30.066567 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:30.066571 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:30.067000 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:30.067142 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:30.067151 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:30.068543 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:30.068549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:30.069685 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:30.069700 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:30.070455 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:30.070460 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:30.070588 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:30.071216 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:30.071246 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:30.074622 (+ 3376us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":231,"threads_started":1}
W20260619 02:51:30.075166 28246 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:30.065697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:53721 (local address 127.18.134.62:39757)
0619 02:51:30.066320 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:30.066325 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:30.066341 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:51:30.066385 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:30.066389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:30.066443 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:30.066538 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:30.067366 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:30.068297 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:30.069845 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:30.070292 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:30.070395 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:30.074905 (+ 4510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.2:53721: BlockingRecv error: recv got EOF from 127.18.134.2:53721 (error 108)
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":411,"threads_started":1}
Jun 19 02:51:30 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:30 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:30.425810 28247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:30.416670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.3:50373)
0619 02:51:30.417161 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:30.417181 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:30.417319 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:30.417762 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:30.417767 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:30.418194 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:30.418336 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:30.418346 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:30.419383 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:30.419389 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:30.420493 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:30.420507 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:30.421366 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:30.421372 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:30.421539 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:30.422193 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:30.422211 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:30.425586 (+ 3375us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":229,"threads_started":1}
W20260619 02:51:30.426081 28246 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:30.416797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:50373 (local address 127.18.134.62:39757)
0619 02:51:30.417179 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:30.417186 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:30.417250 (+ 64us) server_negotiation.cc:407] Connection header received
0619 02:51:30.417446 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:30.417452 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:30.417520 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:30.417722 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:30.418494 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:30.419256 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:30.420650 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:30.421209 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:30.421314 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:30.425813 (+ 4499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.3:50373: BlockingRecv error: recv got EOF from 127.18.134.3:50373 (error 108)
Metrics: {"server-negotiator.queue_time_us":216}
Jun 19 02:51:30 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:30 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:30.546452 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:30.537919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.1:54879)
0619 02:51:30.538441 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:30.538462 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:30.538598 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:30.538951 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:30.538956 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:30.539334 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:30.539486 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:30.539495 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:30.540482 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:30.540487 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:30.541487 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:30.541500 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:30.542265 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:30.542272 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:30.542428 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:30.543009 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:30.543032 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:30.546145 (+ 3113us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":267,"threads_started":1}
W20260619 02:51:30.546592 28246 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:30.538087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:54879 (local address 127.18.134.62:39757)
0619 02:51:30.538376 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:30.538383 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:30.538543 (+ 160us) server_negotiation.cc:407] Connection header received
0619 02:51:30.538736 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:30.538742 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:30.538819 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:30.538904 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:30.539630 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:30.540355 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:30.541631 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:30.542121 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:30.542234 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:30.546340 (+ 4106us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.1:54879: BlockingRecv error: recv got EOF from 127.18.134.1:54879 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Jun 19 02:51:31 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:31 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:31.084733 28249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:31.075834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.2:40435)
0619 02:51:31.076334 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:31.076355 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:31.076464 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:31.076843 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:31.076846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:31.077188 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:31.077375 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:31.077384 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:31.078363 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:31.078367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:31.079416 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:31.079435 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:31.080116 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:31.080120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:31.080232 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:31.080843 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:31.080860 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:31.084368 (+ 3508us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":245,"threads_started":1}
W20260619 02:51:31.084960 28250 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:31.075919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:40435 (local address 127.18.134.62:39757)
0619 02:51:31.076597 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:31.076602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:31.076617 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:51:31.076663 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:31.076666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:31.076720 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:31.076814 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:31.077495 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:31.078213 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:31.079574 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:31.079996 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:31.080099 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:31.084654 (+ 4555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.2:40435: BlockingRecv error: recv got EOF from 127.18.134.2:40435 (error 108)
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":506,"threads_started":1}
Jun 19 02:51:31 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:31 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:31.435643 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:31.426745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.3:42547)
0619 02:51:31.427232 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:31.427250 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:31.427378 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:31.427689 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:31.427693 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:31.428070 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:31.428216 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:31.428225 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:31.429192 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:31.429196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:31.430433 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:31.430448 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:31.431348 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:31.431354 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:31.431516 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:31.432082 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:31.432100 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:31.435326 (+ 3226us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":248,"threads_started":1}
W20260619 02:51:31.435757 28250 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:31.426957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:42547 (local address 127.18.134.62:39757)
0619 02:51:31.427198 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:31.427205 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:31.427319 (+ 114us) server_negotiation.cc:407] Connection header received
0619 02:51:31.427498 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:31.427503 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:31.427565 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:31.427645 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:31.428353 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:31.429055 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:31.430582 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:31.431187 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:31.431298 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:31.435550 (+ 4252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.3:42547: BlockingRecv error: recv got EOF from 127.18.134.3:42547 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
Jun 19 02:51:31 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:31 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:31.556187 28252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:31.547398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.1:57907)
0619 02:51:31.547859 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:31.547880 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:31.547990 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:31.548290 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:31.548294 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:31.548674 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:31.548857 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:31.548866 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:31.549825 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:31.549828 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:31.550838 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:31.550851 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:31.551586 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:31.551591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:31.551709 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:31.552369 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:31.552388 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:31.555975 (+ 3587us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":245,"threads_started":1}
W20260619 02:51:31.556392 28250 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:31.547527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:57907 (local address 127.18.134.62:39757)
0619 02:51:31.547712 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:31.547717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:31.547949 (+ 232us) server_negotiation.cc:407] Connection header received
0619 02:51:31.548080 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:31.548084 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:31.548165 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:31.548272 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:31.548969 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:31.549715 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:31.550993 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:31.551421 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:31.551525 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:31.556177 (+ 4652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.1:57907: BlockingRecv error: recv got EOF from 127.18.134.1:57907 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Jun 19 02:51:32 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:32 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:32.095331 28253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:32.085848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.2:58009)
0619 02:51:32.086346 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:32.086372 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:32.086521 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:32.086862 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:32.086865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:32.087244 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:32.087412 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:32.087420 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:32.088683 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:32.088688 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:32.089816 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:32.089834 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:32.090540 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:32.090546 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:32.090742 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:32.091279 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:32.091296 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:32.095096 (+ 3800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":232,"threads_started":1}
W20260619 02:51:32.095607 28254 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:32.086040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:58009 (local address 127.18.134.62:39757)
0619 02:51:32.086467 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:32.086472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:32.086491 (+ 19us) server_negotiation.cc:407] Connection header received
0619 02:51:32.086624 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:32.086654 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:32.086745 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:32.086838 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:32.087606 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:32.088510 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:32.089976 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:32.090386 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:32.090505 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:32.095344 (+ 4839us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.2:58009: BlockingRecv error: recv got EOF from 127.18.134.2:58009 (error 108)
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":236,"threads_started":1}
Jun 19 02:51:32 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:32 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:32.445271 28255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:32.436625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.3:57169)
0619 02:51:32.437087 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:32.437102 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:32.437213 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:32.437548 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:32.437551 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:32.437958 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:32.438129 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:32.438138 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:32.439170 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:32.439175 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:32.440194 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:32.440204 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:32.441006 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:32.441012 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:32.441170 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:32.441823 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:32.441848 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:32.445047 (+ 3199us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":244,"threads_started":1}
W20260619 02:51:32.445459 28254 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:32.436745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:57169 (local address 127.18.134.62:39757)
0619 02:51:32.436918 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:32.436922 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:32.437175 (+ 253us) server_negotiation.cc:407] Connection header received
0619 02:51:32.437305 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:32.437310 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:32.437374 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:32.437465 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:32.438288 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:32.439049 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:32.440349 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:32.440826 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:32.440927 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:32.445260 (+ 4333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.3:57169: BlockingRecv error: recv got EOF from 127.18.134.3:57169 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Jun 19 02:51:32 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:32 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:32.566138 28256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:32.557283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.1:47195)
0619 02:51:32.557773 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:32.557792 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:32.557904 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:32.558191 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:32.558194 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:32.558603 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:32.558812 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:32.558822 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:32.559915 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:32.559919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:32.560878 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:32.560889 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:32.561703 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:32.561707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:32.561870 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:32.562702 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:32.562725 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:32.565889 (+ 3164us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":259,"threads_started":1}
W20260619 02:51:32.566354 28254 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:32.557350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:47195 (local address 127.18.134.62:39757)
0619 02:51:32.557543 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:32.557550 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:32.557870 (+ 320us) server_negotiation.cc:407] Connection header received
0619 02:51:32.558005 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:32.558009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:32.558071 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:32.558149 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:32.558976 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:32.559780 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:32.561018 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:32.561538 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:32.561646 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:32.566078 (+ 4432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.1:47195: BlockingRecv error: recv got EOF from 127.18.134.1:47195 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 19 02:51:33 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:33 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:33.105444 28257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:33.096210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.2:41695)
0619 02:51:33.096823 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:33.096847 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:33.096966 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:33.097341 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:33.097345 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:33.097758 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:33.097930 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:33.097939 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:33.099112 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:33.099117 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:33.100088 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:33.100102 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:33.100917 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:33.100922 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:33.101080 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:33.101756 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:33.101775 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:33.105094 (+ 3319us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":363,"threads_started":1}
W20260619 02:51:33.105621 28258 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:33.096318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:41695 (local address 127.18.134.62:39757)
0619 02:51:33.096749 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:33.096755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:33.096920 (+ 165us) server_negotiation.cc:407] Connection header received
0619 02:51:33.097104 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:33.097108 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:33.097190 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:33.097309 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:33.098082 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:33.098957 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:33.100284 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:33.100711 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:33.100837 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:33.105373 (+ 4536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.2:41695: BlockingRecv error: recv got EOF from 127.18.134.2:41695 (error 108)
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":227,"threads_started":1}
Jun 19 02:51:33 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:33 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:33.454771 28259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:33.446231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.3:44533)
0619 02:51:33.446792 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:33.446807 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:33.446937 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:33.447305 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:33.447310 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:33.447671 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:33.447827 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:33.447835 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:33.448802 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:33.448807 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:33.449713 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:33.449725 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:33.450444 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:33.450449 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:33.450588 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:33.451338 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:33.451361 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:33.454432 (+ 3071us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":280,"threads_started":1}
W20260619 02:51:33.454867 28258 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:33.446310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:44533 (local address 127.18.134.62:39757)
0619 02:51:33.446511 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:33.446517 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:33.446894 (+ 377us) server_negotiation.cc:407] Connection header received
0619 02:51:33.447053 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:33.447057 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:33.447118 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:33.447199 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:33.447969 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:33.448667 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:33.449846 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:33.450272 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:33.450384 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:33.454659 (+ 4275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.3:44533: BlockingRecv error: recv got EOF from 127.18.134.3:44533 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Jun 19 02:51:33 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:33 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:33.575938 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:33.567129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.1:47323)
0619 02:51:33.567588 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:33.567607 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:33.567722 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:33.568056 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:33.568059 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:33.568452 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:33.568595 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:33.568604 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:33.569636 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:33.569641 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:33.570572 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:33.570595 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:33.571422 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:33.571428 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:33.571598 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:33.572203 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:33.572225 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:33.575605 (+ 3380us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":256,"threads_started":1}
W20260619 02:51:33.576098 28258 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:33.567268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:47323 (local address 127.18.134.62:39757)
0619 02:51:33.567483 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:33.567489 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:33.567682 (+ 193us) server_negotiation.cc:407] Connection header received
0619 02:51:33.567868 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:33.567873 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:33.567934 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:33.568020 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:33.568749 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:33.569489 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:33.570748 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:33.571247 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:33.571371 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:33.575832 (+ 4461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.1:47323: BlockingRecv error: recv got EOF from 127.18.134.1:47323 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Jun 19 02:51:34 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:34 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:34.115041 28261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:34.106532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.2:58413)
0619 02:51:34.107045 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:34.107072 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:34.107202 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:34.107596 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:34.107601 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:34.108003 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:34.108144 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:34.108152 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:34.109134 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:34.109138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:34.110028 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:34.110039 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:34.110749 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:34.110755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:34.110926 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:34.111522 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:34.111540 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:34.114725 (+ 3185us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":282,"threads_started":1}
W20260619 02:51:34.115155 28262 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:34.106701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:58413 (local address 127.18.134.62:39757)
0619 02:51:34.107313 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:34.107319 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:34.107334 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:51:34.107381 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:34.107384 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:34.107445 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:34.107536 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:34.108295 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:34.108993 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:34.110168 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:34.110581 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:34.110729 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:34.114918 (+ 4189us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.2:58413: BlockingRecv error: recv got EOF from 127.18.134.2:58413 (error 108)
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":391,"threads_started":1}
Jun 19 02:51:34 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:34 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.3@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:34.468513 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:34.455791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.3:45411)
0619 02:51:34.456359 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:34.456376 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:34.456506 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:34.456893 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:34.456897 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:34.457392 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:34.457566 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:34.457579 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:34.458750 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:34.458754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:34.459658 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:34.459668 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:34.460411 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:34.460416 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:34.460548 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:34.464903 (+ 4355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:34.464925 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:34.468274 (+ 3349us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":299,"threads_started":1}
W20260619 02:51:34.468843 28262 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:34.455921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:45411 (local address 127.18.134.62:39757)
0619 02:51:34.456119 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:34.456125 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:34.456471 (+ 346us) server_negotiation.cc:407] Connection header received
0619 02:51:34.456643 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:34.456648 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:34.456727 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:34.456840 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:34.457696 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:34.458592 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:34.459810 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:34.460254 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:34.460366 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:34.468588 (+ 8222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.3:45411: BlockingRecv error: recv got EOF from 127.18.134.3:45411 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Jun 19 02:51:34 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:34 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.1@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:34.585629 28264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:34.577153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.1:51407)
0619 02:51:34.577622 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:34.577646 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:34.577787 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:34.578136 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:34.578139 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:34.578519 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:34.578776 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:34.578788 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:34.579802 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:34.579807 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:34.580714 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:34.580727 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:34.581345 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:34.581349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:34.581474 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:34.582014 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:34.582029 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:34.585355 (+ 3326us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":209,"threads_started":1}
W20260619 02:51:34.585736 28262 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:34.577230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:51407 (local address 127.18.134.62:39757)
0619 02:51:34.577521 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:34.577527 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:34.577764 (+ 237us) server_negotiation.cc:407] Connection header received
0619 02:51:34.577894 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:34.577899 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:34.577981 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:34.578064 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:34.578935 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:34.579643 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:34.580851 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:34.581234 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:34.581323 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:34.585533 (+ 4210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.1:51407: BlockingRecv error: recv got EOF from 127.18.134.1:51407 (error 108)
Metrics: {"server-negotiator.queue_time_us":163}
Jun 19 02:51:35 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
Jun 19 02:51:35 dist-test-slave-9mr0 krb5kdc[27204](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.134.2@KRBTEST.COM for oryx/127.18.134.62@KRBTEST.COM, Server not found in Kerberos database
I20260619 02:51:35.124426 28265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:35.116127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39757 (local address 127.18.134.2:46841)
0619 02:51:35.116584 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:35.116607 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:35.116711 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:35.117100 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:35.117103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:35.117444 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:35.117615 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:35.117623 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:35.118649 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:35.118655 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:35.119578 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:35.119590 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:35.120246 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:35.120250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:35.120373 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:35.120932 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:35.120949 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:35.124059 (+ 3110us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.134.62:39757: Server oryx/127.18.134.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":219,"threads_started":1}
W20260619 02:51:35.124567 28266 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:35.116225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:46841 (local address 127.18.134.62:39757)
0619 02:51:35.116853 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:35.116857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:35.116873 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:51:35.116927 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:35.116931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:35.116987 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:35.117072 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:35.117778 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:35.118505 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:35.119723 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:35.120129 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:35.120228 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:35.124338 (+ 4110us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.134.2:46841: BlockingRecv error: recv got EOF from 127.18.134.2:46841 (error 108)
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":424,"threads_started":1}
I20260619 02:51:35.196769 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 27791
I20260619 02:51:35.208873 28268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:35.197885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37374 (local address 127.18.134.1:46743)
0619 02:51:35.198436 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:35.198444 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:35.198470 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:51:35.198533 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:35.198540 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:35.198792 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:35.198967 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:35.200306 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:35.201393 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:35.202611 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:35.203188 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:35.203293 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:35.205834 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:35.205842 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:35.205857 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:35.205895 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:35.208386 (+ 2491us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.134.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0619 02:51:35.208666 (+ 280us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37374: Request ticket server kudu/127.18.134.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":276,"threads_started":1}
W20260619 02:51:35.209097 18968 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 27791: Not authorized: Client connection negotiation failed: client connection to 127.18.134.1:46743: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.134.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260619 02:51:35.209272 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 27791
I20260619 02:51:35.222608 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 27942
I20260619 02:51:35.239773 28269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:35.228388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58960 (local address 127.18.134.2:39677)
0619 02:51:35.229032 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:35.229039 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:35.229071 (+ 32us) server_negotiation.cc:407] Connection header received
0619 02:51:35.229138 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:35.229145 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:35.229393 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:35.229607 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:35.231232 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:35.232347 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:35.233554 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:35.234124 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:35.234224 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:35.236645 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:35.236653 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:35.236666 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:35.236700 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:35.239265 (+ 2565us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.134.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0619 02:51:35.239543 (+ 278us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58960: Request ticket server kudu/127.18.134.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":298,"threads_started":1}
W20260619 02:51:35.239868 28269 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58960: Request ticket server kudu/127.18.134.2@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260619 02:51:35.239967 18968 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 27942: Not authorized: Client connection negotiation failed: client connection to 127.18.134.2:39677: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.134.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260619 02:51:35.240119 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 27942
I20260619 02:51:35.252938 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 28075
I20260619 02:51:35.264864 28270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:35.253945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33060 (local address 127.18.134.3:34549)
0619 02:51:35.254481 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:35.254488 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:35.254512 (+ 24us) server_negotiation.cc:407] Connection header received
0619 02:51:35.254578 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:35.254585 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:35.254867 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:35.255045 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:35.256310 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:35.257282 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:35.258495 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:35.259120 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:35.259245 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:35.261705 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:35.261726 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:35.261741 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:35.261776 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:35.264188 (+ 2412us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.134.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0619 02:51:35.264521 (+ 333us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33060: Request ticket server kudu/127.18.134.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":269,"threads_started":1}
W20260619 02:51:35.264918 18968 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 28075: Not authorized: Client connection negotiation failed: client connection to 127.18.134.3:34549: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.18.134.3@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260619 02:51:35.264972 28270 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33060: Request ticket server kudu/127.18.134.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260619 02:51:35.265069 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 28075
I20260619 02:51:35.277484 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 27722
I20260619 02:51:35.278774 27784 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:35.389384 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 27722
2026-06-19T02:51:35Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (20745 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-19T02:51:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:51:35Z Disabled control of system clock
I20260619 02:51:35.458036 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:37323
--webserver_interface=127.18.134.62
--webserver_port=0
--builtin_ntp_servers=127.18.134.20:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:37323
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 02:51:35.770237 28275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:35.770531 28275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:35.770618 28275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:35.777778 28275 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:51:35.777889 28275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:35.777923 28275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:35.777977 28275 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:51:35.778019 28275 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:51:35.787217 28275 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:37323
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:37323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:35.788441 28275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:35.790105 28275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:35.801405 28280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:35.801465 28281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:35.801362 28283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:35.802004 28275 server_base.cc:1034] running on GCE node
I20260619 02:51:35.802836 28275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:35.804204 28275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:35.805397 28275 hybrid_clock.cc:648] HybridClock initialized: now 1781837495805353 us; error 42 us; skew 500 ppm
I20260619 02:51:35.805724 28275 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:51:35.807940 28275 webserver.cc:460] Webserver started at http://127.18.134.62:43167/ using document root <none> and password file <none>
I20260619 02:51:35.808656 28275 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:35.808753 28275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:35.809054 28275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:51:35.811931 28275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "c727206bae374777bc87fae1e861ab4f"
format_stamp: "Formatted at 2026-06-19 02:51:35 on dist-test-slave-9mr0"
server_key: "b18e9d1783d0783cb7f1213258c19b48"
server_key_iv: "189862057e4245bbc162282f9227ab3a"
server_key_version: "encryptionkey@0"
I20260619 02:51:35.812678 28275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "c727206bae374777bc87fae1e861ab4f"
format_stamp: "Formatted at 2026-06-19 02:51:35 on dist-test-slave-9mr0"
server_key: "b18e9d1783d0783cb7f1213258c19b48"
server_key_iv: "189862057e4245bbc162282f9227ab3a"
server_key_version: "encryptionkey@0"
I20260619 02:51:35.817598 28275 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 02:51:35.820926 28289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:35.822196 28275 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260619 02:51:35.822367 28275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "c727206bae374777bc87fae1e861ab4f"
format_stamp: "Formatted at 2026-06-19 02:51:35 on dist-test-slave-9mr0"
server_key: "b18e9d1783d0783cb7f1213258c19b48"
server_key_iv: "189862057e4245bbc162282f9227ab3a"
server_key_version: "encryptionkey@0"
I20260619 02:51:35.822556 28275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:35.851150 28275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:35.851956 28275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:35.852216 28275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:35.868721 28275 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:37323
I20260619 02:51:35.868741 28340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:37323 every 8 connection(s)
I20260619 02:51:35.870002 28275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:51:35.873533 28341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:35.878860 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 28275
I20260619 02:51:35.879040 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:51:35.879807 18968 external_mini_cluster.cc:1428] Setting key 9ba4b73da9fa52169ddb0b1872ebb162
I20260619 02:51:35.884764 28341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c727206bae374777bc87fae1e861ab4f: Bootstrap starting.
I20260619 02:51:35.888056 28341 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c727206bae374777bc87fae1e861ab4f: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:35.889156 28341 log.cc:824] T 00000000000000000000000000000000 P c727206bae374777bc87fae1e861ab4f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:51:35.892009 28341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c727206bae374777bc87fae1e861ab4f: No bootstrap required, opened a new log
I20260619 02:51:35.892838 28344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:35.881670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33108 (local address 127.18.134.62:37323)
0619 02:51:35.882692 (+ 1022us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:35.882705 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:35.882748 (+ 43us) server_negotiation.cc:407] Connection header received
0619 02:51:35.884327 (+ 1579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:35.884347 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:35.884774 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:35.885104 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:35.885749 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:35.887044 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:35.888751 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:35.889423 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:35.889551 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:35.890779 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:35.890786 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:35.890799 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:51:35.891285 (+ 486us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:35.891354 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:51:35.891373 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:35.891456 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:35.891660 (+ 204us) server_negotiation.cc:299] Negotiation successful
0619 02:51:35.891940 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":345,"threads_started":1}
I20260619 02:51:35.899005 28341 raft_consensus.cc:348] T 00000000000000000000000000000000 P c727206bae374777bc87fae1e861ab4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c727206bae374777bc87fae1e861ab4f" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37323 } }
I20260619 02:51:35.899317 28341 raft_consensus.cc:374] T 00000000000000000000000000000000 P c727206bae374777bc87fae1e861ab4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:35.899410 28341 raft_consensus.cc:729] T 00000000000000000000000000000000 P c727206bae374777bc87fae1e861ab4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c727206bae374777bc87fae1e861ab4f, State: Initialized, Role: FOLLOWER
I20260619 02:51:35.899921 28341 consensus_queue.cc:260] T 00000000000000000000000000000000 P c727206bae374777bc87fae1e861ab4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c727206bae374777bc87fae1e861ab4f" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37323 } }
I20260619 02:51:35.900105 28341 raft_consensus.cc:388] T 00000000000000000000000000000000 P c727206bae374777bc87fae1e861ab4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:51:35.900221 28341 raft_consensus.cc:482] T 00000000000000000000000000000000 P c727206bae374777bc87fae1e861ab4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:51:35.900360 28341 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c727206bae374777bc87fae1e861ab4f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:35.902603 28341 raft_consensus.cc:504] T 00000000000000000000000000000000 P c727206bae374777bc87fae1e861ab4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c727206bae374777bc87fae1e861ab4f" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37323 } }
I20260619 02:51:35.903106 28341 leader_election.cc:302] T 00000000000000000000000000000000 P c727206bae374777bc87fae1e861ab4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c727206bae374777bc87fae1e861ab4f; no voters:
I20260619 02:51:35.903601 28341 leader_election.cc:288] T 00000000000000000000000000000000 P c727206bae374777bc87fae1e861ab4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:51:35.903709 28346 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c727206bae374777bc87fae1e861ab4f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:51:35.904309 28346 raft_consensus.cc:686] T 00000000000000000000000000000000 P c727206bae374777bc87fae1e861ab4f [term 1 LEADER]: Becoming Leader. State: Replica: c727206bae374777bc87fae1e861ab4f, State: Running, Role: LEADER
I20260619 02:51:35.904851 28346 consensus_queue.cc:237] T 00000000000000000000000000000000 P c727206bae374777bc87fae1e861ab4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c727206bae374777bc87fae1e861ab4f" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37323 } }
I20260619 02:51:35.905395 28341 sys_catalog.cc:564] T 00000000000000000000000000000000 P c727206bae374777bc87fae1e861ab4f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:51:35.907418 28348 sys_catalog.cc:455] T 00000000000000000000000000000000 P c727206bae374777bc87fae1e861ab4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c727206bae374777bc87fae1e861ab4f. Latest consensus state: current_term: 1 leader_uuid: "c727206bae374777bc87fae1e861ab4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c727206bae374777bc87fae1e861ab4f" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37323 } } }
I20260619 02:51:35.907867 28348 sys_catalog.cc:458] T 00000000000000000000000000000000 P c727206bae374777bc87fae1e861ab4f [sys.catalog]: This master's current role is: LEADER
I20260619 02:51:35.908085 28347 sys_catalog.cc:455] T 00000000000000000000000000000000 P c727206bae374777bc87fae1e861ab4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c727206bae374777bc87fae1e861ab4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c727206bae374777bc87fae1e861ab4f" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37323 } } }
I20260619 02:51:35.908494 28347 sys_catalog.cc:458] T 00000000000000000000000000000000 P c727206bae374777bc87fae1e861ab4f [sys.catalog]: This master's current role is: LEADER
I20260619 02:51:35.909926 28352 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:51:35.914785 28352 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:51:35.923264 28352 catalog_manager.cc:1347] Generated new cluster ID: 659e0f4fa142488c9cec9224beb0bee1
I20260619 02:51:35.923378 28352 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:51:35.955516 28352 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:51:35.956799 28352 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:51:35.971828 28352 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c727206bae374777bc87fae1e861ab4f: Generated new TSK 0
I20260619 02:51:35.972692 28352 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:51:35.986259 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:0
--local_ip_for_outbound_sockets=127.18.134.1
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:37323
--builtin_ntp_servers=127.18.134.20:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 02:51:36.297690 28365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:36.297951 28365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:36.298058 28365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:36.305202 28365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:36.305300 28365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:36.305430 28365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:51:36.314515 28365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:37323
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:36.315800 28365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:36.317499 28365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:36.330250 28371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:36.330216 28373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:36.330278 28370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:36.331008 28365 server_base.cc:1034] running on GCE node
I20260619 02:51:36.331562 28365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:36.332461 28365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:36.333654 28365 hybrid_clock.cc:648] HybridClock initialized: now 1781837496333605 us; error 40 us; skew 500 ppm
I20260619 02:51:36.333951 28365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:51:36.336043 28365 webserver.cc:460] Webserver started at http://127.18.134.1:45043/ using document root <none> and password file <none>
I20260619 02:51:36.336697 28365 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:36.336805 28365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:36.337106 28365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:51:36.339869 28365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/data/instance:
uuid: "b48e219e1a7c4744b689111ca865b382"
format_stamp: "Formatted at 2026-06-19 02:51:36 on dist-test-slave-9mr0"
server_key: "47c0c430a0d94f1a1f1a55edf59cbc7d"
server_key_iv: "7e9c91bf09b6e1c0cccff8f390462439"
server_key_version: "encryptionkey@0"
I20260619 02:51:36.340603 28365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance:
uuid: "b48e219e1a7c4744b689111ca865b382"
format_stamp: "Formatted at 2026-06-19 02:51:36 on dist-test-slave-9mr0"
server_key: "47c0c430a0d94f1a1f1a55edf59cbc7d"
server_key_iv: "7e9c91bf09b6e1c0cccff8f390462439"
server_key_version: "encryptionkey@0"
I20260619 02:51:36.345672 28365 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:51:36.349118 28379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:36.350499 28365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:51:36.350687 28365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "b48e219e1a7c4744b689111ca865b382"
format_stamp: "Formatted at 2026-06-19 02:51:36 on dist-test-slave-9mr0"
server_key: "47c0c430a0d94f1a1f1a55edf59cbc7d"
server_key_iv: "7e9c91bf09b6e1c0cccff8f390462439"
server_key_version: "encryptionkey@0"
I20260619 02:51:36.350914 28365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:36.364050 28365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:36.364876 28365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:36.365149 28365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:36.366076 28365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:51:36.367547 28365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:51:36.367641 28365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:36.367748 28365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:51:36.367816 28365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:36.396098 28365 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:36407
I20260619 02:51:36.396157 28491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:36407 every 8 connection(s)
I20260619 02:51:36.397342 28365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:51:36.399286 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 28365
I20260619 02:51:36.399452 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance
I20260619 02:51:36.400182 18968 external_mini_cluster.cc:1428] Setting key 6deaee1a8af3653035307fc7dfb69657
I20260619 02:51:36.402891 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:0
--local_ip_for_outbound_sockets=127.18.134.2
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:37323
--builtin_ntp_servers=127.18.134.20:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 02:51:36.410418 28494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:36.399842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:38611 (local address 127.18.134.62:37323)
0619 02:51:36.400268 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:36.400274 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:36.401598 (+ 1324us) server_negotiation.cc:407] Connection header received
0619 02:51:36.403519 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:36.403525 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:36.403599 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:36.403709 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:36.404733 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:36.405630 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:36.407360 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:36.407886 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:36.408001 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:36.409592 (+ 1591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:36.409599 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:36.409604 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:51:36.409633 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:36.409681 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:51:36.409697 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:36.409808 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:36.410040 (+ 232us) server_negotiation.cc:299] Negotiation successful
0619 02:51:36.410139 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":200,"threads_started":1}
I20260619 02:51:36.411757 28495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:36.400428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:37323 (local address 127.18.134.1:38611)
0619 02:51:36.401439 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:36.401485 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:36.403265 (+ 1780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:36.403909 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:36.403917 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:36.404004 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:36.404583 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:36.404600 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:36.405798 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:36.405802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:36.407216 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:36.407229 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:36.408115 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:36.408121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:36.408341 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:36.409083 (+ 742us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:51:36.409087 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:36.409157 (+ 70us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:51:36.409264 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:36.409822 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:36.409840 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:36.409847 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 02:51:36.410008 (+ 161us) client_negotiation.cc:240] Negotiation successful
0619 02:51:36.410411 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":294,"threads_started":1}
I20260619 02:51:36.413277 28492 heartbeater.cc:347] Connected to a master server at 127.18.134.62:37323
I20260619 02:51:36.413585 28492 heartbeater.cc:464] Registering TS with master...
I20260619 02:51:36.414449 28492 heartbeater.cc:511] Master 127.18.134.62:37323 requested a full tablet report, sending...
I20260619 02:51:36.416625 28306 ts_manager.cc:194] Registered new tserver with Master: b48e219e1a7c4744b689111ca865b382 (127.18.134.1:36407)
I20260619 02:51:36.418129 28306 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.134.1:38611
W20260619 02:51:36.721313 28496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:36.721596 28496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:36.721698 28496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:36.729058 28496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:36.729167 28496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:36.729310 28496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:51:36.738411 28496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:37323
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:36.739704 28496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:36.741459 28496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:36.754428 28501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:36.754359 28504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:36.754428 28502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:36.754897 28496 server_base.cc:1034] running on GCE node
I20260619 02:51:36.755676 28496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:36.756690 28496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:36.757912 28496 hybrid_clock.cc:648] HybridClock initialized: now 1781837496757854 us; error 45 us; skew 500 ppm
I20260619 02:51:36.758204 28496 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:51:36.760543 28496 webserver.cc:460] Webserver started at http://127.18.134.2:33113/ using document root <none> and password file <none>
I20260619 02:51:36.761241 28496 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:36.761356 28496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:36.761672 28496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:51:36.764561 28496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/data/instance:
uuid: "ae002a4e17da4a8ba7dc4e7f650aaf53"
format_stamp: "Formatted at 2026-06-19 02:51:36 on dist-test-slave-9mr0"
server_key: "14696a42aef0e9389bacf5f20981c661"
server_key_iv: "e8d7884ba7b60d923e68cbfbbabb9d1a"
server_key_version: "encryptionkey@0"
I20260619 02:51:36.765311 28496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance:
uuid: "ae002a4e17da4a8ba7dc4e7f650aaf53"
format_stamp: "Formatted at 2026-06-19 02:51:36 on dist-test-slave-9mr0"
server_key: "14696a42aef0e9389bacf5f20981c661"
server_key_iv: "e8d7884ba7b60d923e68cbfbbabb9d1a"
server_key_version: "encryptionkey@0"
I20260619 02:51:36.770557 28496 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 02:51:36.774093 28510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:36.775552 28496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:51:36.775691 28496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "ae002a4e17da4a8ba7dc4e7f650aaf53"
format_stamp: "Formatted at 2026-06-19 02:51:36 on dist-test-slave-9mr0"
server_key: "14696a42aef0e9389bacf5f20981c661"
server_key_iv: "e8d7884ba7b60d923e68cbfbbabb9d1a"
server_key_version: "encryptionkey@0"
I20260619 02:51:36.775887 28496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:36.795815 28496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:36.796622 28496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:36.796877 28496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:36.797804 28496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:51:36.799365 28496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:51:36.799453 28496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:36.799531 28496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:51:36.799583 28496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:36.828886 28496 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:36107
I20260619 02:51:36.828954 28622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:36107 every 8 connection(s)
I20260619 02:51:36.830143 28496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:51:36.836398 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 28496
I20260619 02:51:36.836535 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance
I20260619 02:51:36.836949 18968 external_mini_cluster.cc:1428] Setting key 3e43406884dac312b186dfd823abec4b
I20260619 02:51:36.839704 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:0
--local_ip_for_outbound_sockets=127.18.134.3
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:37323
--builtin_ntp_servers=127.18.134.20:35067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 02:51:36.842855 28494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:36.832603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:48073 (local address 127.18.134.62:37323)
0619 02:51:36.832780 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:36.832784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:36.834143 (+ 1359us) server_negotiation.cc:407] Connection header received
0619 02:51:36.835868 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:36.835873 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:36.835957 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:36.836041 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:36.837162 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:36.837973 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:36.840151 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:36.840581 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:36.840686 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:36.842137 (+ 1451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:36.842142 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:36.842146 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:51:36.842162 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:36.842193 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:51:36.842204 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:36.842283 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:36.842485 (+ 202us) server_negotiation.cc:299] Negotiation successful
0619 02:51:36.842563 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 02:51:36.844125 28625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:36.833099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:37323 (local address 127.18.134.2:48073)
0619 02:51:36.833990 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:36.834031 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:36.835609 (+ 1578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:36.836282 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:36.836291 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:36.836380 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:36.836998 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:36.837023 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:36.838162 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:36.838168 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:36.839987 (+ 1819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:36.840002 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:36.840798 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:36.840804 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:36.841096 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:36.841801 (+ 705us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:51:36.841804 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:36.841854 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:51:36.841964 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:36.842283 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:36.842294 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:36.842301 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 02:51:36.842468 (+ 167us) client_negotiation.cc:240] Negotiation successful
0619 02:51:36.842894 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":280,"threads_started":1}
I20260619 02:51:36.845470 28623 heartbeater.cc:347] Connected to a master server at 127.18.134.62:37323
I20260619 02:51:36.845763 28623 heartbeater.cc:464] Registering TS with master...
I20260619 02:51:36.846624 28623 heartbeater.cc:511] Master 127.18.134.62:37323 requested a full tablet report, sending...
I20260619 02:51:36.848187 28306 ts_manager.cc:194] Registered new tserver with Master: ae002a4e17da4a8ba7dc4e7f650aaf53 (127.18.134.2:36107)
I20260619 02:51:36.849020 28306 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.134.2:48073
W20260619 02:51:37.161854 28626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:37.162166 28626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:37.162307 28626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:37.169514 28626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:37.169626 28626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:37.169749 28626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:51:37.178937 28626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:35067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:37323
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:37.180294 28626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:37.181979 28626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:37.194526 28631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:37.194561 28632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:37.194489 28634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:37.195441 28626 server_base.cc:1034] running on GCE node
I20260619 02:51:37.196030 28626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:37.197036 28626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:37.198243 28626 hybrid_clock.cc:648] HybridClock initialized: now 1781837497198187 us; error 42 us; skew 500 ppm
I20260619 02:51:37.198588 28626 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:51:37.200863 28626 webserver.cc:460] Webserver started at http://127.18.134.3:40021/ using document root <none> and password file <none>
I20260619 02:51:37.201519 28626 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:37.201609 28626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:37.201875 28626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:51:37.204684 28626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/data/instance:
uuid: "8698f68652d340be9b111f376c6189b2"
format_stamp: "Formatted at 2026-06-19 02:51:37 on dist-test-slave-9mr0"
server_key: "dcec748cf1c45eb99e9d1b8e26d9c437"
server_key_iv: "f25f157cb10a1aa99adace7aa8859fff"
server_key_version: "encryptionkey@0"
I20260619 02:51:37.205457 28626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance:
uuid: "8698f68652d340be9b111f376c6189b2"
format_stamp: "Formatted at 2026-06-19 02:51:37 on dist-test-slave-9mr0"
server_key: "dcec748cf1c45eb99e9d1b8e26d9c437"
server_key_iv: "f25f157cb10a1aa99adace7aa8859fff"
server_key_version: "encryptionkey@0"
I20260619 02:51:37.210594 28626 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 02:51:37.214030 28640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:37.215431 28626 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:51:37.215596 28626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "8698f68652d340be9b111f376c6189b2"
format_stamp: "Formatted at 2026-06-19 02:51:37 on dist-test-slave-9mr0"
server_key: "dcec748cf1c45eb99e9d1b8e26d9c437"
server_key_iv: "f25f157cb10a1aa99adace7aa8859fff"
server_key_version: "encryptionkey@0"
I20260619 02:51:37.215821 28626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:37.239705 28626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:37.240573 28626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:37.240855 28626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:37.241816 28626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:51:37.243330 28626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:51:37.243415 28626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:37.243520 28626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:51:37.243593 28626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:37.272351 28626 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:35115
I20260619 02:51:37.272428 28752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:35115 every 8 connection(s)
I20260619 02:51:37.273612 28626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:51:37.283128 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 28626
I20260619 02:51:37.283259 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance
I20260619 02:51:37.283676 18968 external_mini_cluster.cc:1428] Setting key f6c65ea6dbee7493b4b731a40cf3ee1d
I20260619 02:51:37.285677 28494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:37.276158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:36459 (local address 127.18.134.62:37323)
0619 02:51:37.276324 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:37.276328 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:37.277664 (+ 1336us) server_negotiation.cc:407] Connection header received
0619 02:51:37.279466 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:37.279472 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:37.279552 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:37.279637 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:37.280694 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:37.281409 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:37.282929 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:37.283385 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:37.283491 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:37.284906 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:37.284912 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:37.284917 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:51:37.284937 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:37.284976 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:51:37.284992 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:37.285084 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:37.285268 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 02:51:37.285343 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 02:51:37.286840 28755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:37.276631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:37323 (local address 127.18.134.3:36459)
0619 02:51:37.277505 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:37.277545 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:37.279201 (+ 1656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:37.279883 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:37.279892 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:37.279985 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:37.280536 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:37.280552 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:37.281551 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:37.281555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:37.282800 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:37.282811 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:37.283591 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:37.283597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:37.283835 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:37.284520 (+ 685us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:51:37.284524 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:37.284589 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:51:37.284688 (+ 99us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:37.285082 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:37.285090 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:37.285095 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 02:51:37.285236 (+ 141us) client_negotiation.cc:240] Negotiation successful
0619 02:51:37.285582 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":266,"threads_started":1}
I20260619 02:51:37.288154 28753 heartbeater.cc:347] Connected to a master server at 127.18.134.62:37323
I20260619 02:51:37.288426 28753 heartbeater.cc:464] Registering TS with master...
I20260619 02:51:37.289091 28753 heartbeater.cc:511] Master 127.18.134.62:37323 requested a full tablet report, sending...
I20260619 02:51:37.290303 28306 ts_manager.cc:194] Registered new tserver with Master: 8698f68652d340be9b111f376c6189b2 (127.18.134.3:35115)
I20260619 02:51:37.291217 28306 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.134.3:36459
I20260619 02:51:37.299100 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 02:51:37.304895 28494 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:37.303487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33112 (local address 127.18.134.62:37323)
0619 02:51:37.303634 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:37.303638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:37.304111 (+ 473us) server_negotiation.cc:407] Connection header received
0619 02:51:37.304250 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:37.304254 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:37.304333 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:37.304414 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:37.304703 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33112: BlockingRecv error: recv got EOF from 127.0.0.1:33112 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 02:51:37.305514 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 28365
I20260619 02:51:37.313048 28762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:37.306349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37786 (local address 127.18.134.1:36407)
0619 02:51:37.306876 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:37.306882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:37.306906 (+ 24us) server_negotiation.cc:407] Connection header received
0619 02:51:37.306963 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:37.306968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:37.307159 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:37.307307 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:37.307691 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:37.308652 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:37.309884 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:37.310438 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:37.310550 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:37.311646 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:37.311657 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:37.311674 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:51:37.312257 (+ 583us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:37.312304 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:51:37.312328 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:37.312434 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:37.312571 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 02:51:37.312715 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":286,"threads_started":1}
I20260619 02:51:37.314285 28487 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:37.393708 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 28365
I20260619 02:51:37.406548 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 28496
I20260619 02:51:37.414088 28764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:37.407505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50492 (local address 127.18.134.2:36107)
0619 02:51:37.407934 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:37.407941 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:37.407966 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:51:37.408027 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:37.408034 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:37.408236 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:37.408402 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:37.408765 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:37.409701 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:37.410993 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:37.411536 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:37.411665 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:37.412670 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:37.412677 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:37.412691 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:51:37.413245 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:37.413304 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:51:37.413328 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:37.413410 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:37.413604 (+ 194us) server_negotiation.cc:299] Negotiation successful
0619 02:51:37.413754 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":204,"threads_started":1}
I20260619 02:51:37.415396 28618 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:37.494140 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 28496
I20260619 02:51:37.506084 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 28626
I20260619 02:51:37.513703 28766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:37.507014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37204 (local address 127.18.134.3:35115)
0619 02:51:37.507609 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:37.507616 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:37.507642 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:51:37.507697 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:37.507703 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:37.507925 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:37.508063 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:37.508495 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:37.509493 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:37.510712 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:37.511260 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:37.511373 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:37.512350 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:37.512358 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:37.512370 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:51:37.512931 (+ 561us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:37.512975 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:51:37.512994 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:37.513088 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:37.513245 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 02:51:37.513373 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":304,"threads_started":1}
I20260619 02:51:37.514978 28748 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:37.594683 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 28626
I20260619 02:51:37.607152 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 28275
I20260619 02:51:37.608224 28336 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:37.705571 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 28275
2026-06-19T02:51:37Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2311 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-19T02:51:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:51:37Z Disabled control of system clock
I20260619 02:51:37.767831 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:34539
--webserver_interface=127.18.134.62
--webserver_port=0
--builtin_ntp_servers=127.18.134.20:46079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:34539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 02:51:38.080092 28772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:38.080395 28772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:38.080483 28772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:38.087685 28772 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:51:38.087796 28772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:38.087832 28772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:38.087890 28772 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:51:38.087934 28772 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:51:38.097097 28772 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:46079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:34539
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:34539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:38.098315 28772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:38.100092 28772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:38.111729 28778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:38.112035 28780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:38.111722 28777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:38.112612 28772 server_base.cc:1034] running on GCE node
I20260619 02:51:38.113473 28772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:38.115017 28772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:38.116243 28772 hybrid_clock.cc:648] HybridClock initialized: now 1781837498116195 us; error 43 us; skew 500 ppm
I20260619 02:51:38.116595 28772 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:51:38.118947 28772 webserver.cc:460] Webserver started at http://127.18.134.62:42409/ using document root <none> and password file <none>
I20260619 02:51:38.119678 28772 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:38.119778 28772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:38.120088 28772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:51:38.123010 28772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "82278c1e0884459492811181a1419029"
format_stamp: "Formatted at 2026-06-19 02:51:38 on dist-test-slave-9mr0"
server_key: "c07f6f13fe03e17f4829d8a0e5eb3336"
server_key_iv: "addff04741e546bc777e2d55afc911c8"
server_key_version: "encryptionkey@0"
I20260619 02:51:38.123785 28772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "82278c1e0884459492811181a1419029"
format_stamp: "Formatted at 2026-06-19 02:51:38 on dist-test-slave-9mr0"
server_key: "c07f6f13fe03e17f4829d8a0e5eb3336"
server_key_iv: "addff04741e546bc777e2d55afc911c8"
server_key_version: "encryptionkey@0"
I20260619 02:51:38.128811 28772 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:51:38.132192 28786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:38.133539 28772 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 02:51:38.133697 28772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "82278c1e0884459492811181a1419029"
format_stamp: "Formatted at 2026-06-19 02:51:38 on dist-test-slave-9mr0"
server_key: "c07f6f13fe03e17f4829d8a0e5eb3336"
server_key_iv: "addff04741e546bc777e2d55afc911c8"
server_key_version: "encryptionkey@0"
I20260619 02:51:38.133894 28772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:38.139506 28772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:38.140239 28772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:38.140506 28772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:38.157872 28772 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:34539
I20260619 02:51:38.157922 28837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:34539 every 8 connection(s)
I20260619 02:51:38.159224 28772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:51:38.162951 28838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:38.167542 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 28772
I20260619 02:51:38.167810 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:51:38.168443 18968 external_mini_cluster.cc:1428] Setting key ea554539d429cb556203f28acfc1191c
I20260619 02:51:38.173396 28838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82278c1e0884459492811181a1419029: Bootstrap starting.
I20260619 02:51:38.176705 28841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:38.170489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33184 (local address 127.18.134.62:34539)
0619 02:51:38.171600 (+ 1111us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:38.171615 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:38.171662 (+ 47us) server_negotiation.cc:407] Connection header received
0619 02:51:38.173301 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:38.173323 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:38.173763 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:38.174094 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:38.175190 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:38.175198 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:38.175213 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:51:38.175229 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:38.175305 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:51:38.175325 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:38.175495 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:38.175595 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 02:51:38.175855 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":385,"threads_started":1}
I20260619 02:51:38.177125 28838 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82278c1e0884459492811181a1419029: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:38.178339 28838 log.cc:824] T 00000000000000000000000000000000 P 82278c1e0884459492811181a1419029: Log is configured to *not* fsync() on all Append() calls
I20260619 02:51:38.181208 28838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82278c1e0884459492811181a1419029: No bootstrap required, opened a new log
I20260619 02:51:38.188061 28838 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82278c1e0884459492811181a1419029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82278c1e0884459492811181a1419029" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 34539 } }
I20260619 02:51:38.188411 28838 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82278c1e0884459492811181a1419029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:38.188503 28838 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82278c1e0884459492811181a1419029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82278c1e0884459492811181a1419029, State: Initialized, Role: FOLLOWER
I20260619 02:51:38.189138 28838 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82278c1e0884459492811181a1419029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82278c1e0884459492811181a1419029" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 34539 } }
I20260619 02:51:38.189368 28838 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82278c1e0884459492811181a1419029 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:51:38.189486 28838 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82278c1e0884459492811181a1419029 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:51:38.189631 28838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82278c1e0884459492811181a1419029 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:38.191795 28838 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82278c1e0884459492811181a1419029 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82278c1e0884459492811181a1419029" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 34539 } }
I20260619 02:51:38.192265 28838 leader_election.cc:302] T 00000000000000000000000000000000 P 82278c1e0884459492811181a1419029 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82278c1e0884459492811181a1419029; no voters:
I20260619 02:51:38.192831 28838 leader_election.cc:288] T 00000000000000000000000000000000 P 82278c1e0884459492811181a1419029 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:51:38.192957 28843 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82278c1e0884459492811181a1419029 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:51:38.193590 28843 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82278c1e0884459492811181a1419029 [term 1 LEADER]: Becoming Leader. State: Replica: 82278c1e0884459492811181a1419029, State: Running, Role: LEADER
I20260619 02:51:38.194098 28843 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82278c1e0884459492811181a1419029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82278c1e0884459492811181a1419029" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 34539 } }
I20260619 02:51:38.194826 28838 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82278c1e0884459492811181a1419029 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:51:38.196751 28844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82278c1e0884459492811181a1419029 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82278c1e0884459492811181a1419029" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82278c1e0884459492811181a1419029" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 34539 } } }
I20260619 02:51:38.197098 28844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82278c1e0884459492811181a1419029 [sys.catalog]: This master's current role is: LEADER
I20260619 02:51:38.197540 28845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82278c1e0884459492811181a1419029 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82278c1e0884459492811181a1419029. Latest consensus state: current_term: 1 leader_uuid: "82278c1e0884459492811181a1419029" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82278c1e0884459492811181a1419029" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 34539 } } }
I20260619 02:51:38.197832 28845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82278c1e0884459492811181a1419029 [sys.catalog]: This master's current role is: LEADER
I20260619 02:51:38.199285 28850 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:51:38.204636 28850 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:51:38.213102 28850 catalog_manager.cc:1347] Generated new cluster ID: 050cb658877f47faa1bec779f60380c4
I20260619 02:51:38.213254 28850 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:51:38.229808 28850 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:51:38.230280 28850 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:51:38.239128 28850 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82278c1e0884459492811181a1419029: Generated new TSK 0
I20260619 02:51:38.240144 28850 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:51:38.252260 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:0
--local_ip_for_outbound_sockets=127.18.134.1
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:34539
--builtin_ntp_servers=127.18.134.20:46079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 02:51:38.573308 28862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:38.573603 28862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:38.573693 28862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:38.581673 28862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:38.581799 28862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:38.581910 28862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:51:38.591238 28862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:46079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:34539
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:38.592521 28862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:38.594214 28862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:38.607162 28867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:38.607194 28870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:38.607127 28868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:38.608004 28862 server_base.cc:1034] running on GCE node
I20260619 02:51:38.608614 28862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:38.609557 28862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:38.610733 28862 hybrid_clock.cc:648] HybridClock initialized: now 1781837498610665 us; error 61 us; skew 500 ppm
I20260619 02:51:38.611053 28862 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:51:38.613402 28862 webserver.cc:460] Webserver started at http://127.18.134.1:33579/ using document root <none> and password file <none>
I20260619 02:51:38.614096 28862 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:38.614197 28862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:38.614506 28862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:51:38.617482 28862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/data/instance:
uuid: "beaccf73c4684712941e84c44e8d7779"
format_stamp: "Formatted at 2026-06-19 02:51:38 on dist-test-slave-9mr0"
server_key: "37e2a629e8d64ae02cf39df4798e5676"
server_key_iv: "f1f3bd935e375be1f434af76b29ecfbc"
server_key_version: "encryptionkey@0"
I20260619 02:51:38.618288 28862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance:
uuid: "beaccf73c4684712941e84c44e8d7779"
format_stamp: "Formatted at 2026-06-19 02:51:38 on dist-test-slave-9mr0"
server_key: "37e2a629e8d64ae02cf39df4798e5676"
server_key_iv: "f1f3bd935e375be1f434af76b29ecfbc"
server_key_version: "encryptionkey@0"
I20260619 02:51:38.623647 28862 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260619 02:51:38.627226 28876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:38.628624 28862 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:51:38.628783 28862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "beaccf73c4684712941e84c44e8d7779"
format_stamp: "Formatted at 2026-06-19 02:51:38 on dist-test-slave-9mr0"
server_key: "37e2a629e8d64ae02cf39df4798e5676"
server_key_iv: "f1f3bd935e375be1f434af76b29ecfbc"
server_key_version: "encryptionkey@0"
I20260619 02:51:38.628994 28862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:38.634497 28862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:38.635325 28862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:38.635572 28862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:38.636495 28862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:51:38.637902 28862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:51:38.637995 28862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:38.638098 28862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:51:38.638163 28862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:38.667779 28862 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:41869
I20260619 02:51:38.667840 28988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:41869 every 8 connection(s)
I20260619 02:51:38.669039 28862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:51:38.673506 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 28862
I20260619 02:51:38.673725 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance
I20260619 02:51:38.674471 18968 external_mini_cluster.cc:1428] Setting key 1dc88c03c2fc60ca06d9b7de53a47c5c
I20260619 02:51:38.677491 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:0
--local_ip_for_outbound_sockets=127.18.134.2
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:34539
--builtin_ntp_servers=127.18.134.20:46079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260619 02:51:38.678483 28841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:38.671593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:43193 (local address 127.18.134.62:34539)
0619 02:51:38.671788 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:38.671795 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:38.673195 (+ 1400us) server_negotiation.cc:407] Connection header received
0619 02:51:38.675443 (+ 2248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:38.675450 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:38.675526 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:38.675674 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:38.677805 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:38.677810 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:38.677813 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:51:38.677831 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:38.677865 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:51:38.677876 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:38.677983 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:38.678197 (+ 214us) server_negotiation.cc:299] Negotiation successful
0619 02:51:38.678273 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:51:38.679823 28991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:38.672128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:34539 (local address 127.18.134.1:43193)
0619 02:51:38.673022 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:38.673070 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:38.675183 (+ 2113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:38.676446 (+ 1263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:38.676459 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:38.676593 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:38.677344 (+ 751us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:51:38.677348 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:38.677406 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:51:38.677529 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:38.677970 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:38.677979 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:38.677987 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 02:51:38.678277 (+ 290us) client_negotiation.cc:240] Negotiation successful
0619 02:51:38.678565 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":263,"threads_started":1}
I20260619 02:51:38.681315 28989 heartbeater.cc:347] Connected to a master server at 127.18.134.62:34539
I20260619 02:51:38.681593 28989 heartbeater.cc:464] Registering TS with master...
I20260619 02:51:38.682169 28989 heartbeater.cc:511] Master 127.18.134.62:34539 requested a full tablet report, sending...
I20260619 02:51:38.684240 28803 ts_manager.cc:194] Registered new tserver with Master: beaccf73c4684712941e84c44e8d7779 (127.18.134.1:41869)
W20260619 02:51:39.000092 28992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:39.000406 28992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:39.000497 28992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:39.008021 28992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:39.008145 28992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:39.008260 28992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:51:39.017524 28992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:46079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:34539
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:39.018841 28992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:39.020622 28992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:39.033396 28997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:39.034162 28998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:39.035018 29000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:39.035427 28992 server_base.cc:1034] running on GCE node
I20260619 02:51:39.036022 28992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:39.036954 28992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:39.038197 28992 hybrid_clock.cc:648] HybridClock initialized: now 1781837499038109 us; error 102 us; skew 500 ppm
I20260619 02:51:39.038561 28992 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:51:39.041234 28992 webserver.cc:460] Webserver started at http://127.18.134.2:40753/ using document root <none> and password file <none>
I20260619 02:51:39.041913 28992 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:39.041999 28992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:39.042280 28992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:51:39.045190 28992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/data/instance:
uuid: "a11e92dad8304d558d567835d45e791a"
format_stamp: "Formatted at 2026-06-19 02:51:39 on dist-test-slave-9mr0"
server_key: "30e79983261fd516a52c5eff9c3cdead"
server_key_iv: "892ae1c97035b273861d8401623f35b7"
server_key_version: "encryptionkey@0"
I20260619 02:51:39.045955 28992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance:
uuid: "a11e92dad8304d558d567835d45e791a"
format_stamp: "Formatted at 2026-06-19 02:51:39 on dist-test-slave-9mr0"
server_key: "30e79983261fd516a52c5eff9c3cdead"
server_key_iv: "892ae1c97035b273861d8401623f35b7"
server_key_version: "encryptionkey@0"
I20260619 02:51:39.051298 28992 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260619 02:51:39.054975 29006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:39.056778 28992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:51:39.056983 28992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "a11e92dad8304d558d567835d45e791a"
format_stamp: "Formatted at 2026-06-19 02:51:39 on dist-test-slave-9mr0"
server_key: "30e79983261fd516a52c5eff9c3cdead"
server_key_iv: "892ae1c97035b273861d8401623f35b7"
server_key_version: "encryptionkey@0"
I20260619 02:51:39.057255 28992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:39.064328 28992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:39.065168 28992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:39.065426 28992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:39.066421 28992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:51:39.068007 28992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:51:39.068097 28992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:39.068202 28992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:51:39.068259 28992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:39.099114 28992 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:46121
I20260619 02:51:39.099282 29118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:46121 every 8 connection(s)
I20260619 02:51:39.100600 28992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:51:39.108547 28841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:39.103164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:35751 (local address 127.18.134.62:34539)
0619 02:51:39.103347 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:39.103351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:39.104737 (+ 1386us) server_negotiation.cc:407] Connection header received
0619 02:51:39.106369 (+ 1632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:39.106374 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:39.106431 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:39.106521 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:39.107890 (+ 1369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:39.107894 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:39.107897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:51:39.107913 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:39.107944 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:51:39.107955 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:39.108060 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:39.108281 (+ 221us) server_negotiation.cc:299] Negotiation successful
0619 02:51:39.108334 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 02:51:39.109383 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 28992
I20260619 02:51:39.109488 29121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:39.103707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:34539 (local address 127.18.134.2:35751)
0619 02:51:39.104611 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:39.104646 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:39.106112 (+ 1466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:39.106836 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:39.106845 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:39.106955 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:39.107613 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:51:39.107616 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:39.107652 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:51:39.107742 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:39.108062 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:39.108070 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:39.108076 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 02:51:39.108337 (+ 261us) client_negotiation.cc:240] Negotiation successful
0619 02:51:39.108552 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":260,"threads_started":1}
I20260619 02:51:39.109561 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance
I20260619 02:51:39.110107 18968 external_mini_cluster.cc:1428] Setting key 1acdb3a90c35ff3c8f0674d5b616f487
I20260619 02:51:39.111054 29119 heartbeater.cc:347] Connected to a master server at 127.18.134.62:34539
I20260619 02:51:39.111394 29119 heartbeater.cc:464] Registering TS with master...
I20260619 02:51:39.112098 29119 heartbeater.cc:511] Master 127.18.134.62:34539 requested a full tablet report, sending...
I20260619 02:51:39.112915 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:0
--local_ip_for_outbound_sockets=127.18.134.3
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:34539
--builtin_ntp_servers=127.18.134.20:46079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260619 02:51:39.113586 28803 ts_manager.cc:194] Registered new tserver with Master: a11e92dad8304d558d567835d45e791a (127.18.134.2:46121)
W20260619 02:51:39.434229 29122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:39.434496 29122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:39.434619 29122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:39.442000 29122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:39.442112 29122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:39.442261 29122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:51:39.451332 29122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:46079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:34539
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:39.452596 29122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:39.454348 29122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:39.466753 29128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:39.466787 29130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:39.466938 29127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:39.467659 29122 server_base.cc:1034] running on GCE node
I20260619 02:51:39.468178 29122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:39.469100 29122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:39.470324 29122 hybrid_clock.cc:648] HybridClock initialized: now 1781837499470266 us; error 50 us; skew 500 ppm
I20260619 02:51:39.470649 29122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:51:39.482579 29122 webserver.cc:460] Webserver started at http://127.18.134.3:40123/ using document root <none> and password file <none>
I20260619 02:51:39.483389 29122 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:39.483501 29122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:39.483803 29122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:51:39.486573 29122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/data/instance:
uuid: "6234967275894a7e9d814f726387503a"
format_stamp: "Formatted at 2026-06-19 02:51:39 on dist-test-slave-9mr0"
server_key: "a98e3e192d4b6a14d3b860fd063ae653"
server_key_iv: "ce9bd9a14c78ef7e86edc117d11726dc"
server_key_version: "encryptionkey@0"
I20260619 02:51:39.487382 29122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance:
uuid: "6234967275894a7e9d814f726387503a"
format_stamp: "Formatted at 2026-06-19 02:51:39 on dist-test-slave-9mr0"
server_key: "a98e3e192d4b6a14d3b860fd063ae653"
server_key_iv: "ce9bd9a14c78ef7e86edc117d11726dc"
server_key_version: "encryptionkey@0"
I20260619 02:51:39.492250 29122 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260619 02:51:39.495597 29136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:39.496862 29122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 02:51:39.497004 29122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "6234967275894a7e9d814f726387503a"
format_stamp: "Formatted at 2026-06-19 02:51:39 on dist-test-slave-9mr0"
server_key: "a98e3e192d4b6a14d3b860fd063ae653"
server_key_iv: "ce9bd9a14c78ef7e86edc117d11726dc"
server_key_version: "encryptionkey@0"
I20260619 02:51:39.497195 29122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:39.502512 29122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:39.503247 29122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:39.503484 29122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:39.504279 29122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:51:39.505543 29122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:51:39.505635 29122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:39.505733 29122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:51:39.505797 29122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:39.534049 29122 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:35451
I20260619 02:51:39.534096 29248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:35451 every 8 connection(s)
I20260619 02:51:39.535317 29122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:51:39.542786 28841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:39.537611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:34021 (local address 127.18.134.62:34539)
0619 02:51:39.537779 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:39.537784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:39.539212 (+ 1428us) server_negotiation.cc:407] Connection header received
0619 02:51:39.540816 (+ 1604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:39.540822 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:39.540890 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:39.540970 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:39.542082 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:39.542088 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:39.542093 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:51:39.542109 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:39.542137 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:51:39.542149 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:39.542245 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:39.542443 (+ 198us) server_negotiation.cc:299] Negotiation successful
0619 02:51:39.542505 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 02:51:39.543602 29251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:39.538108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:34539 (local address 127.18.134.3:34021)
0619 02:51:39.539075 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:39.539108 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:39.540578 (+ 1470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:39.541184 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:39.541192 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:39.541275 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:39.541820 (+ 545us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:51:39.541823 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:39.541859 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:51:39.541928 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:39.542228 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:39.542237 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:39.542245 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 02:51:39.542498 (+ 253us) client_negotiation.cc:240] Negotiation successful
0619 02:51:39.542753 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":278,"threads_started":1}
I20260619 02:51:39.543845 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 29122
I20260619 02:51:39.544024 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance
I20260619 02:51:39.544497 18968 external_mini_cluster.cc:1428] Setting key 83a414330761403ef9924ad72c10cc79
I20260619 02:51:39.544811 29249 heartbeater.cc:347] Connected to a master server at 127.18.134.62:34539
I20260619 02:51:39.545070 29249 heartbeater.cc:464] Registering TS with master...
I20260619 02:51:39.545604 29249 heartbeater.cc:511] Master 127.18.134.62:34539 requested a full tablet report, sending...
I20260619 02:51:39.547024 28802 ts_manager.cc:194] Registered new tserver with Master: 6234967275894a7e9d814f726387503a (127.18.134.3:35451)
I20260619 02:51:39.559590 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 02:51:39.565439 28841 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:51:39.564084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33196 (local address 127.18.134.62:34539)
0619 02:51:39.564326 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:39.564333 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:39.564650 (+ 317us) server_negotiation.cc:407] Connection header received
0619 02:51:39.564799 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:39.564802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:39.564861 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:39.564937 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:39.565192 (+ 255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33196: BlockingRecv error: recv got EOF from 127.0.0.1:33196 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 02:51:39.566176 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 28862
I20260619 02:51:39.569334 29258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:39.567161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44608 (local address 127.18.134.1:41869)
0619 02:51:39.567643 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:39.567652 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:39.567678 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:51:39.567733 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:39.567739 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:39.567940 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:39.568061 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:39.568782 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:39.568788 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:39.568798 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:51:39.568815 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:39.568851 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:51:39.568864 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:39.568955 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:39.569025 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 02:51:39.569099 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":230,"threads_started":1}
I20260619 02:51:39.570430 28984 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:39.648820 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 28862
I20260619 02:51:39.663575 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 28992
I20260619 02:51:39.666929 29260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:39.664427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42298 (local address 127.18.134.2:46121)
0619 02:51:39.664937 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:39.664943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:39.664969 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:51:39.665029 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:39.665035 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:39.665236 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:39.665358 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:39.666228 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:39.666235 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:39.666246 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:51:39.666263 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:39.666304 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:51:39.666317 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:39.666413 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:39.666531 (+ 118us) server_negotiation.cc:299] Negotiation successful
0619 02:51:39.666611 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":247,"threads_started":1}
I20260619 02:51:39.668195 29114 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:39.746840 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 28992
I20260619 02:51:39.758754 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 29122
I20260619 02:51:39.762451 29262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:39.759786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58736 (local address 127.18.134.3:35451)
0619 02:51:39.760453 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:39.760460 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:39.760489 (+ 29us) server_negotiation.cc:407] Connection header received
0619 02:51:39.760546 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:39.760552 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:39.760774 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:39.760895 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:39.761831 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:39.761837 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:39.761847 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:51:39.761859 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:39.761897 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:51:39.761909 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:39.762031 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:39.762075 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 02:51:39.762150 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":360,"threads_started":1}
I20260619 02:51:39.763619 29244 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:39.843237 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 29122
I20260619 02:51:39.856873 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 28772
I20260619 02:51:39.857999 28833 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:39.956416 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 28772
2026-06-19T02:51:39Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2251 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:51:40 dist-test-slave-9mr0 krb5kdc[29267](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:51:40 dist-test-slave-9mr0 krb5kdc[29267](info): set up 2 sockets
Jun 19 02:51:40 dist-test-slave-9mr0 krb5kdc[29267](info): commencing operation
krb5kdc: starting...
W20260619 02:51:40.975360 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.982s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:51:41 dist-test-slave-9mr0 krb5kdc[29267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:51:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:51:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:51:41.195770 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:40705
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:36475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:40705
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:51:41.502373 29280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:41.502660 29280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:41.502769 29280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:41.509800 29280 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:51:41.509912 29280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:41.509955 29280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:41.510010 29280 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:51:41.510051 29280 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:51:41.519732 29280 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:36475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:40705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:40705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:41.520944 29280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:41.522145 29280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:41.533030 29285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:41.533110 29288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:41.533077 29286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:41.533811 29280 server_base.cc:1034] running on GCE node
I20260619 02:51:41.534538 29280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:41.535944 29280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:41.537178 29280 hybrid_clock.cc:648] HybridClock initialized: now 1781837501537119 us; error 49 us; skew 500 ppm
I20260619 02:51:41.537504 29280 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:51:41 dist-test-slave-9mr0 krb5kdc[29267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837501, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:51:41.540496 29280 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:51:41.541894 29280 webserver.cc:460] Webserver started at http://127.18.134.62:36613/ using document root <none> and password file <none>
I20260619 02:51:41.542534 29280 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:41.542670 29280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:41.542944 29280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:51:41.545666 29280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "3c8da4dfb75443faac1678e18b79216d"
format_stamp: "Formatted at 2026-06-19 02:51:41 on dist-test-slave-9mr0"
server_key: "acec50ce55cae172f67b932806dec27a"
server_key_iv: "8480ba3f33c08fa81ec4ac8c4a552620"
server_key_version: "encryptionkey@0"
I20260619 02:51:41.546360 29280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "3c8da4dfb75443faac1678e18b79216d"
format_stamp: "Formatted at 2026-06-19 02:51:41 on dist-test-slave-9mr0"
server_key: "acec50ce55cae172f67b932806dec27a"
server_key_iv: "8480ba3f33c08fa81ec4ac8c4a552620"
server_key_version: "encryptionkey@0"
I20260619 02:51:41.551167 29280 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 02:51:41.554515 29295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:41.555759 29280 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:51:41.555912 29280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "3c8da4dfb75443faac1678e18b79216d"
format_stamp: "Formatted at 2026-06-19 02:51:41 on dist-test-slave-9mr0"
server_key: "acec50ce55cae172f67b932806dec27a"
server_key_iv: "8480ba3f33c08fa81ec4ac8c4a552620"
server_key_version: "encryptionkey@0"
I20260619 02:51:41.556080 29280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:41.569640 29280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:41.573787 29280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:41.574038 29280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:41.590726 29280 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:40705
I20260619 02:51:41.590777 29346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:40705 every 8 connection(s)
I20260619 02:51:41.592051 29280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:51:41.595337 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 29280
I20260619 02:51:41.595575 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:51:41.595942 29347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:41.596355 18968 external_mini_cluster.cc:1428] Setting key 86c67ae47fe0cb58dc51b9022cf4e850
I20260619 02:51:41.610680 29347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c8da4dfb75443faac1678e18b79216d: Bootstrap starting.
Jun 19 02:51:41 dist-test-slave-9mr0 krb5kdc[29267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:51:41.613708 29347 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c8da4dfb75443faac1678e18b79216d: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:41.614737 29347 log.cc:824] T 00000000000000000000000000000000 P 3c8da4dfb75443faac1678e18b79216d: Log is configured to *not* fsync() on all Append() calls
I20260619 02:51:41.617331 29347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c8da4dfb75443faac1678e18b79216d: No bootstrap required, opened a new log
I20260619 02:51:41.618606 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:41.598349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52296 (local address 127.18.134.62:40705)
0619 02:51:41.599189 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:41.599211 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:41.599251 (+ 40us) server_negotiation.cc:407] Connection header received
0619 02:51:41.600893 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:41.600917 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:41.601345 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:41.601686 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:41.603710 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:41.604990 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:41.606827 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:41.607547 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:41.607692 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:41.612059 (+ 4367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:41.612066 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:41.612082 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:41.612160 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:41.614994 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:41.615605 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:41.615615 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:41.615622 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:41.615777 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:41.616125 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:41.616129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:41.616131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:41.616610 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:41.616883 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:41.617279 (+ 396us) server_negotiation.cc:299] Negotiation successful
0619 02:51:41.617690 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":357,"threads_started":1}
I20260619 02:51:41.625101 29347 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c8da4dfb75443faac1678e18b79216d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c8da4dfb75443faac1678e18b79216d" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 40705 } }
I20260619 02:51:41.625468 29347 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c8da4dfb75443faac1678e18b79216d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:41.625567 29347 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c8da4dfb75443faac1678e18b79216d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c8da4dfb75443faac1678e18b79216d, State: Initialized, Role: FOLLOWER
I20260619 02:51:41.626116 29347 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c8da4dfb75443faac1678e18b79216d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c8da4dfb75443faac1678e18b79216d" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 40705 } }
I20260619 02:51:41.626332 29347 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c8da4dfb75443faac1678e18b79216d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:51:41.626451 29347 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c8da4dfb75443faac1678e18b79216d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:51:41.626595 29347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c8da4dfb75443faac1678e18b79216d [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:41.628648 29347 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c8da4dfb75443faac1678e18b79216d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c8da4dfb75443faac1678e18b79216d" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 40705 } }
I20260619 02:51:41.629096 29347 leader_election.cc:302] T 00000000000000000000000000000000 P 3c8da4dfb75443faac1678e18b79216d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c8da4dfb75443faac1678e18b79216d; no voters:
I20260619 02:51:41.629631 29347 leader_election.cc:288] T 00000000000000000000000000000000 P 3c8da4dfb75443faac1678e18b79216d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:51:41.629755 29352 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c8da4dfb75443faac1678e18b79216d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:51:41.630360 29352 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c8da4dfb75443faac1678e18b79216d [term 1 LEADER]: Becoming Leader. State: Replica: 3c8da4dfb75443faac1678e18b79216d, State: Running, Role: LEADER
I20260619 02:51:41.630980 29352 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c8da4dfb75443faac1678e18b79216d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c8da4dfb75443faac1678e18b79216d" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 40705 } }
I20260619 02:51:41.631616 29347 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c8da4dfb75443faac1678e18b79216d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:51:41.633586 29353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c8da4dfb75443faac1678e18b79216d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c8da4dfb75443faac1678e18b79216d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c8da4dfb75443faac1678e18b79216d" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 40705 } } }
I20260619 02:51:41.633965 29353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c8da4dfb75443faac1678e18b79216d [sys.catalog]: This master's current role is: LEADER
I20260619 02:51:41.633612 29354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c8da4dfb75443faac1678e18b79216d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c8da4dfb75443faac1678e18b79216d. Latest consensus state: current_term: 1 leader_uuid: "3c8da4dfb75443faac1678e18b79216d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c8da4dfb75443faac1678e18b79216d" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 40705 } } }
I20260619 02:51:41.634953 29354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c8da4dfb75443faac1678e18b79216d [sys.catalog]: This master's current role is: LEADER
I20260619 02:51:41.635687 29360 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:51:41.640636 29360 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:51:41.649788 29360 catalog_manager.cc:1347] Generated new cluster ID: dbec967254ab45159468e6fc0aa84eba
I20260619 02:51:41.649906 29360 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:51:41.668597 29360 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:51:41.669910 29360 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:51:41.675371 29360 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c8da4dfb75443faac1678e18b79216d: Generated new TSK 0
I20260619 02:51:41.676144 29360 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.134.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
I20260619 02:51:41.738530 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:0
--local_ip_for_outbound_sockets=127.18.134.1
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:40705
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:36475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:51:42.077008 29373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:42.077286 29373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:42.077378 29373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:42.084506 29373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:42.084618 29373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:42.084738 29373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:51:42.094337 29373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:36475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:40705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:42.095566 29373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:42.096814 29373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:42.108875 29378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:42.108912 29379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:42.108806 29381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:42.109722 29373 server_base.cc:1034] running on GCE node
I20260619 02:51:42.110252 29373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:42.111142 29373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:42.112355 29373 hybrid_clock.cc:648] HybridClock initialized: now 1781837502112312 us; error 36 us; skew 500 ppm
I20260619 02:51:42.112634 29373 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:51:42 dist-test-slave-9mr0 krb5kdc[29267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837502, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:51:42.115654 29373 init.cc:377] Logged in from keytab as kudu/127.18.134.1@KRBTEST.COM (short username kudu)
I20260619 02:51:42.117074 29373 webserver.cc:460] Webserver started at http://127.18.134.1:38949/ using document root <none> and password file <none>
I20260619 02:51:42.117720 29373 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:42.117830 29373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:42.118101 29373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:51:42.120865 29373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/data/instance:
uuid: "d36667b8064843d9bc0f90a61276d4db"
format_stamp: "Formatted at 2026-06-19 02:51:42 on dist-test-slave-9mr0"
server_key: "fce7c6681602f4521d5ce3d57670fc38"
server_key_iv: "d69c5bb38d7dadfa974492a9b6d646e1"
server_key_version: "encryptionkey@0"
I20260619 02:51:42.121568 29373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance:
uuid: "d36667b8064843d9bc0f90a61276d4db"
format_stamp: "Formatted at 2026-06-19 02:51:42 on dist-test-slave-9mr0"
server_key: "fce7c6681602f4521d5ce3d57670fc38"
server_key_iv: "d69c5bb38d7dadfa974492a9b6d646e1"
server_key_version: "encryptionkey@0"
I20260619 02:51:42.126461 29373 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260619 02:51:42.129825 29388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:42.131206 29373 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:51:42.131372 29373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "d36667b8064843d9bc0f90a61276d4db"
format_stamp: "Formatted at 2026-06-19 02:51:42 on dist-test-slave-9mr0"
server_key: "fce7c6681602f4521d5ce3d57670fc38"
server_key_iv: "d69c5bb38d7dadfa974492a9b6d646e1"
server_key_version: "encryptionkey@0"
I20260619 02:51:42.131543 29373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:42.156972 29373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:42.161192 29373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:42.161464 29373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:42.162385 29373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:51:42.163693 29373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:51:42.163767 29373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:42.163872 29373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:51:42.163923 29373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:42.193603 29373 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:36019
I20260619 02:51:42.193650 29500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:36019 every 8 connection(s)
I20260619 02:51:42.194902 29373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:51:42.202456 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 29373
I20260619 02:51:42.202670 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance
I20260619 02:51:42.203261 18968 external_mini_cluster.cc:1428] Setting key d6cdec423c28de783776c9ff5c5ad612
Jun 19 02:51:42 dist-test-slave-9mr0 krb5kdc[29267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837502, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:51:42.214202 29503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:42.197223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:34721 (local address 127.18.134.62:40705)
0619 02:51:42.197625 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:42.197629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:42.198460 (+ 831us) server_negotiation.cc:407] Connection header received
0619 02:51:42.200223 (+ 1763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:42.200229 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:42.200307 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:42.200406 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:42.201837 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:42.202528 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:42.204175 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:42.204813 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:42.204928 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:42.208410 (+ 3482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:42.208416 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:42.208420 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:42.208456 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:42.211316 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:42.212013 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:42.212018 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:42.212022 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:42.212161 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:42.212624 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:42.212627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:42.212629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:42.212887 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:42.212992 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:42.213789 (+ 797us) server_negotiation.cc:299] Negotiation successful
0619 02:51:42.213950 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":207,"threads_started":1}
I20260619 02:51:42.215574 29504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:42.197618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:40705 (local address 127.18.134.1:34721)
0619 02:51:42.198296 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:42.198348 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:42.199966 (+ 1618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:42.200631 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:42.200638 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:42.201119 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:42.201665 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:42.201682 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:42.202698 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:42.202702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:42.203994 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:42.204013 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:42.205033 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:42.205039 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:42.205258 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:42.205937 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:42.205958 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:42.208183 (+ 2225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:42.211440 (+ 3257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:42.211446 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:42.211459 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:42.211887 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:42.212275 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:42.212280 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:42.212284 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:42.212524 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:42.212995 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:42.213003 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:42.213420 (+ 417us) client_negotiation.cc:769] Sending connection context
0619 02:51:42.213693 (+ 273us) client_negotiation.cc:240] Negotiation successful
0619 02:51:42.214210 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":278,"threads_started":1}
I20260619 02:51:42.217001 29501 heartbeater.cc:347] Connected to a master server at 127.18.134.62:40705
I20260619 02:51:42.217310 29501 heartbeater.cc:464] Registering TS with master...
I20260619 02:51:42.218104 29501 heartbeater.cc:511] Master 127.18.134.62:40705 requested a full tablet report, sending...
I20260619 02:51:42.220373 29312 ts_manager.cc:194] Registered new tserver with Master: d36667b8064843d9bc0f90a61276d4db (127.18.134.1:36019)
I20260619 02:51:42.221968 29312 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.1@KRBTEST.COM'} at 127.18.134.1:34721
WARNING: no policy specified for kudu/127.18.134.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
I20260619 02:51:42.251116 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:0
--local_ip_for_outbound_sockets=127.18.134.2
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:40705
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:36475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:51:42.560463 29507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:42.560777 29507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:42.560907 29507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:42.568109 29507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:42.568241 29507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:42.568398 29507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:51:42.578099 29507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:36475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:40705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:42.579404 29507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:42.580651 29507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:42.592712 29512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:42.593286 29513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:42.593411 29515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:42.593905 29507 server_base.cc:1034] running on GCE node
I20260619 02:51:42.594480 29507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:42.595414 29507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:42.596627 29507 hybrid_clock.cc:648] HybridClock initialized: now 1781837502596576 us; error 47 us; skew 500 ppm
I20260619 02:51:42.596946 29507 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:51:42 dist-test-slave-9mr0 krb5kdc[29267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837502, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:51:42.600152 29507 init.cc:377] Logged in from keytab as kudu/127.18.134.2@KRBTEST.COM (short username kudu)
I20260619 02:51:42.601648 29507 webserver.cc:460] Webserver started at http://127.18.134.2:45667/ using document root <none> and password file <none>
I20260619 02:51:42.602361 29507 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:42.602475 29507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:42.602794 29507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:51:42.605592 29507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/data/instance:
uuid: "f26a9f62d2324ece8439899a2fa1236a"
format_stamp: "Formatted at 2026-06-19 02:51:42 on dist-test-slave-9mr0"
server_key: "5f7d3d36aa39f92df11482c57c2cb655"
server_key_iv: "6f220f994721c722832f6a238f041288"
server_key_version: "encryptionkey@0"
I20260619 02:51:42.606321 29507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance:
uuid: "f26a9f62d2324ece8439899a2fa1236a"
format_stamp: "Formatted at 2026-06-19 02:51:42 on dist-test-slave-9mr0"
server_key: "5f7d3d36aa39f92df11482c57c2cb655"
server_key_iv: "6f220f994721c722832f6a238f041288"
server_key_version: "encryptionkey@0"
I20260619 02:51:42.611555 29507 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260619 02:51:42.614976 29522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:42.616309 29507 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260619 02:51:42.616461 29507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "f26a9f62d2324ece8439899a2fa1236a"
format_stamp: "Formatted at 2026-06-19 02:51:42 on dist-test-slave-9mr0"
server_key: "5f7d3d36aa39f92df11482c57c2cb655"
server_key_iv: "6f220f994721c722832f6a238f041288"
server_key_version: "encryptionkey@0"
I20260619 02:51:42.616647 29507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:42.639940 29507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:42.643848 29507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:42.644079 29507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:42.644954 29507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:51:42.646237 29507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:51:42.646311 29507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:42.646390 29507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:51:42.646431 29507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:42.676039 29507 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:36899
I20260619 02:51:42.676127 29634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:36899 every 8 connection(s)
I20260619 02:51:42.677241 29507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:51:42.680980 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 29507
I20260619 02:51:42.681173 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance
I20260619 02:51:42.681557 18968 external_mini_cluster.cc:1428] Setting key 7557171c8013d307db3ea8ef56069c7f
Jun 19 02:51:42 dist-test-slave-9mr0 krb5kdc[29267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837502, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:51:42.695717 29503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:42.679555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:41639 (local address 127.18.134.62:40705)
0619 02:51:42.679721 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:42.679725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:42.680842 (+ 1117us) server_negotiation.cc:407] Connection header received
0619 02:51:42.682698 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:42.682703 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:42.682782 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:42.682871 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:42.684337 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:42.685020 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:42.686486 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:42.686918 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:42.687024 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:42.690225 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:42.690230 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:42.690233 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:42.690264 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:42.692782 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:42.693481 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:42.693486 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:42.693488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:42.693607 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:42.694021 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:42.694025 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:42.694029 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:42.694292 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:42.694412 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:42.695156 (+ 744us) server_negotiation.cc:299] Negotiation successful
0619 02:51:42.695374 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 02:51:42.696688 29637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:42.679972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:40705 (local address 127.18.134.2:41639)
0619 02:51:42.680686 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:42.680722 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:42.682448 (+ 1726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:42.683122 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:42.683133 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:42.683596 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:42.684178 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:42.684200 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:42.685161 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:42.685167 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:42.686355 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:42.686367 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:42.687134 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:42.687141 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:42.687393 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:42.687997 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:42.688018 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:42.690010 (+ 1992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:42.692927 (+ 2917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:42.692937 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:42.692954 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:42.693368 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:42.693732 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:42.693735 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:42.693738 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:42.693916 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:42.694421 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:42.694429 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:42.694807 (+ 378us) client_negotiation.cc:769] Sending connection context
0619 02:51:42.695049 (+ 242us) client_negotiation.cc:240] Negotiation successful
0619 02:51:42.695574 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":269,"threads_started":1}
I20260619 02:51:42.698065 29635 heartbeater.cc:347] Connected to a master server at 127.18.134.62:40705
I20260619 02:51:42.698388 29635 heartbeater.cc:464] Registering TS with master...
I20260619 02:51:42.699112 29635 heartbeater.cc:511] Master 127.18.134.62:40705 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.134.3@KRBTEST.COM; defaulting to no policy
I20260619 02:51:42.700467 29312 ts_manager.cc:194] Registered new tserver with Master: f26a9f62d2324ece8439899a2fa1236a (127.18.134.2:36899)
I20260619 02:51:42.701289 29312 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.2@KRBTEST.COM'} at 127.18.134.2:41639
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
I20260619 02:51:42.727577 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:0
--local_ip_for_outbound_sockets=127.18.134.3
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:40705
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:36475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:51:43.043398 29640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:51:43.043699 29640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:51:43.043825 29640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:51:43.050968 29640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:51:43.051095 29640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:51:43.051251 29640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:51:43.063583 29640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:36475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:40705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:51:43.064831 29640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:51:43.066073 29640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:51:43.078918 29646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:43.078884 29648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:51:43.078881 29645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:51:43.079787 29640 server_base.cc:1034] running on GCE node
I20260619 02:51:43.080437 29640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:51:43.081413 29640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:51:43.082621 29640 hybrid_clock.cc:648] HybridClock initialized: now 1781837503082574 us; error 39 us; skew 500 ppm
I20260619 02:51:43.082970 29640 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:51:43 dist-test-slave-9mr0 krb5kdc[29267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837503, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:51:43.086104 29640 init.cc:377] Logged in from keytab as kudu/127.18.134.3@KRBTEST.COM (short username kudu)
I20260619 02:51:43.087601 29640 webserver.cc:460] Webserver started at http://127.18.134.3:44983/ using document root <none> and password file <none>
I20260619 02:51:43.088279 29640 fs_manager.cc:359] Metadata directory not provided
I20260619 02:51:43.088398 29640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:51:43.088675 29640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:51:43.091550 29640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/data/instance:
uuid: "c7002d3773974376bc5650372ef558b4"
format_stamp: "Formatted at 2026-06-19 02:51:43 on dist-test-slave-9mr0"
server_key: "112e1273a99f7acadd3af94af156e780"
server_key_iv: "2410c8e2e2c25ea1087f0f9599fdbc72"
server_key_version: "encryptionkey@0"
I20260619 02:51:43.092366 29640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance:
uuid: "c7002d3773974376bc5650372ef558b4"
format_stamp: "Formatted at 2026-06-19 02:51:43 on dist-test-slave-9mr0"
server_key: "112e1273a99f7acadd3af94af156e780"
server_key_iv: "2410c8e2e2c25ea1087f0f9599fdbc72"
server_key_version: "encryptionkey@0"
I20260619 02:51:43.097332 29640 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 02:51:43.100704 29655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:43.102167 29640 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:51:43.102322 29640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "c7002d3773974376bc5650372ef558b4"
format_stamp: "Formatted at 2026-06-19 02:51:43 on dist-test-slave-9mr0"
server_key: "112e1273a99f7acadd3af94af156e780"
server_key_iv: "2410c8e2e2c25ea1087f0f9599fdbc72"
server_key_version: "encryptionkey@0"
I20260619 02:51:43.102514 29640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:51:43.116710 29640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:51:43.120702 29640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:51:43.120959 29640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:51:43.121826 29640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:51:43.123087 29640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:51:43.123172 29640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:43.123284 29640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:51:43.123339 29640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:51:43.151259 29640 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:41551
I20260619 02:51:43.151291 29767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:41551 every 8 connection(s)
I20260619 02:51:43.152458 29640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:51:43.157877 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 29640
I20260619 02:51:43.158073 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance
I20260619 02:51:43.158475 18968 external_mini_cluster.cc:1428] Setting key 3b04385983b550e0f710d360db7ccdaa
Jun 19 02:51:43 dist-test-slave-9mr0 krb5kdc[29267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837503, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:51:43.171532 29503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:43.154892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:38265 (local address 127.18.134.62:40705)
0619 02:51:43.155051 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:43.155056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:43.156174 (+ 1118us) server_negotiation.cc:407] Connection header received
0619 02:51:43.157942 (+ 1768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:43.157948 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:43.158066 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:43.158171 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:43.159990 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.160810 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.162321 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.162773 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.162879 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:43.166183 (+ 3304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:43.166188 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:43.166191 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:43.166239 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:43.168700 (+ 2461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:43.169380 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:43.169384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:43.169386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:43.169512 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:43.169942 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:43.169946 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:43.169948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:43.170266 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:43.170384 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:43.171125 (+ 741us) server_negotiation.cc:299] Negotiation successful
0619 02:51:43.171287 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 02:51:43.172734 29770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:43.155287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:40705 (local address 127.18.134.3:38265)
0619 02:51:43.156002 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:43.156046 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:43.157671 (+ 1625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:43.158442 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:43.158450 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:43.159061 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:51:43.159804 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:43.159825 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.160957 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.160963 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:43.162186 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:43.162197 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.162990 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.162997 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:43.163231 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:43.163867 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:51:43.163889 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:51:43.165952 (+ 2063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:51:43.168840 (+ 2888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:43.168847 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:43.168860 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:43.169273 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:43.169613 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:51:43.169617 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:51:43.169619 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:51:43.169798 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:51:43.170408 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:51:43.170414 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:51:43.170772 (+ 358us) client_negotiation.cc:769] Sending connection context
0619 02:51:43.171020 (+ 248us) client_negotiation.cc:240] Negotiation successful
0619 02:51:43.171463 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":297,"threads_started":1}
I20260619 02:51:43.174064 29768 heartbeater.cc:347] Connected to a master server at 127.18.134.62:40705
I20260619 02:51:43.174348 29768 heartbeater.cc:464] Registering TS with master...
I20260619 02:51:43.175074 29768 heartbeater.cc:511] Master 127.18.134.62:40705 requested a full tablet report, sending...
I20260619 02:51:43.176394 29312 ts_manager.cc:194] Registered new tserver with Master: c7002d3773974376bc5650372ef558b4 (127.18.134.3:41551)
I20260619 02:51:43.177157 29312 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.3@KRBTEST.COM'} at 127.18.134.3:38265
I20260619 02:51:43.185848 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:51:43.201815 29503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:43.189993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52306 (local address 127.18.134.62:40705)
0619 02:51:43.190188 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:43.190195 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:43.190569 (+ 374us) server_negotiation.cc:407] Connection header received
0619 02:51:43.190733 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:43.190736 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:43.190794 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:43.190870 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:43.192194 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.192942 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.194200 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.194869 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.194995 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:43.197083 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:43.197090 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:43.197094 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:43.197132 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:43.199657 (+ 2525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:43.200238 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:43.200241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:43.200244 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:43.200362 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:43.200654 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:43.200657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:43.200659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:43.200953 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:43.201122 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:43.201367 (+ 245us) server_negotiation.cc:299] Negotiation successful
0619 02:51:43.201517 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 02:51:43.205741 29312 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52306:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:51:43.212494 29312 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:51:43.225433 29501 heartbeater.cc:503] Master 127.18.134.62:40705 was elected leader, sending a full tablet report...
I20260619 02:51:43.240851 29783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:43.224416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:41551 (local address 127.0.0.1:50374)
0619 02:51:43.228944 (+ 4528us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:43.228999 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:43.229182 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:43.232940 (+ 3758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:43.232944 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:43.232968 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:43.233379 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:43.233391 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.235492 (+ 2101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.235497 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:43.237997 (+ 2500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:43.238013 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.239722 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.239728 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:43.239963 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:43.240019 (+ 56us) client_negotiation.cc:769] Sending connection context
0619 02:51:43.240140 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 02:51:43.240331 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4258,"mutex_wait_us":15,"thread_start_us":7647,"threads_started":1}
I20260619 02:51:43.241529 29780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:43.224462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50374 (local address 127.18.134.3:41551)
0619 02:51:43.231017 (+ 6555us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:43.231026 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:43.231056 (+ 30us) server_negotiation.cc:407] Connection header received
0619 02:51:43.231152 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:43.231159 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:43.231453 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:43.233584 (+ 2131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:43.233844 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.235351 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.238138 (+ 2787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.239515 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.239666 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:43.239901 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:43.241074 (+ 1173us) server_negotiation.cc:299] Negotiation successful
0619 02:51:43.241239 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6394,"thread_start_us":6057,"threads_started":1}
I20260619 02:51:43.241531 29781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:43.225117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:36019 (local address 127.0.0.1:38488)
0619 02:51:43.230227 (+ 5110us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:43.230246 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:43.231675 (+ 1429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:43.232376 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:43.232381 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:43.232402 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:43.232685 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:43.232693 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.234623 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.234654 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:43.238281 (+ 3627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:43.238293 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.240747 (+ 2454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.240754 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:43.240950 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:43.240982 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 02:51:43.241055 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 02:51:43.241182 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4984,"mutex_wait_us":31,"thread_start_us":3414,"threads_started":1}
I20260619 02:51:43.241837 29782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:43.224904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38488 (local address 127.18.134.1:36019)
0619 02:51:43.229837 (+ 4933us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:43.229846 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:43.230344 (+ 498us) server_negotiation.cc:407] Connection header received
0619 02:51:43.231859 (+ 1515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:43.231865 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:43.232036 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:43.232151 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:43.232910 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.234468 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.238450 (+ 3982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.240250 (+ 1800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.240382 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:43.240599 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:43.241393 (+ 794us) server_negotiation.cc:299] Negotiation successful
0619 02:51:43.241551 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4741,"thread_start_us":4701,"threads_started":1}
I20260619 02:51:43.245357 29436 tablet_service.cc:1467] Processing CreateTablet for tablet 4482dcfa907546b9bbaca2fc165d5b7a (DEFAULT_TABLE table=test-table [id=bc2cdde458f248c38f7d08bd2a3d6f47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:51:43.246093 29784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:43.229471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:36899 (local address 127.0.0.1:57072)
0619 02:51:43.239357 (+ 9886us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:43.239375 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:43.239506 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:43.240377 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:43.240382 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:43.240407 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:43.240851 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:43.240862 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.242597 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.242602 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:43.243925 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:43.243937 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.245575 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.245578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:43.245683 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:43.245704 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 02:51:43.245771 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 02:51:43.245865 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9758,"thread_start_us":9636,"threads_started":1}
I20260619 02:51:43.246315 29785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:43.229683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57072 (local address 127.18.134.2:36899)
0619 02:51:43.238957 (+ 9274us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:43.238965 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:43.239482 (+ 517us) server_negotiation.cc:407] Connection header received
0619 02:51:43.239644 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:43.239654 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:43.239934 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:43.240136 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:43.241029 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.242453 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.244076 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.245408 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.245543 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:43.245718 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:43.245806 (+ 88us) server_negotiation.cc:299] Negotiation successful
0619 02:51:43.245951 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9089,"thread_start_us":8338,"threads_started":1}
I20260619 02:51:43.246907 29436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4482dcfa907546b9bbaca2fc165d5b7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:43.248553 29703 tablet_service.cc:1467] Processing CreateTablet for tablet 4482dcfa907546b9bbaca2fc165d5b7a (DEFAULT_TABLE table=test-table [id=bc2cdde458f248c38f7d08bd2a3d6f47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:51:43.249185 29570 tablet_service.cc:1467] Processing CreateTablet for tablet 4482dcfa907546b9bbaca2fc165d5b7a (DEFAULT_TABLE table=test-table [id=bc2cdde458f248c38f7d08bd2a3d6f47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:51:43.249635 29703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4482dcfa907546b9bbaca2fc165d5b7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:43.250128 29570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4482dcfa907546b9bbaca2fc165d5b7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:51:43.260421 29786 tablet_bootstrap.cc:492] T 4482dcfa907546b9bbaca2fc165d5b7a P f26a9f62d2324ece8439899a2fa1236a: Bootstrap starting.
I20260619 02:51:43.262425 29787 tablet_bootstrap.cc:492] T 4482dcfa907546b9bbaca2fc165d5b7a P c7002d3773974376bc5650372ef558b4: Bootstrap starting.
I20260619 02:51:43.264778 29786 tablet_bootstrap.cc:654] T 4482dcfa907546b9bbaca2fc165d5b7a P f26a9f62d2324ece8439899a2fa1236a: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:43.266026 29787 tablet_bootstrap.cc:654] T 4482dcfa907546b9bbaca2fc165d5b7a P c7002d3773974376bc5650372ef558b4: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:43.266057 29786 log.cc:824] T 4482dcfa907546b9bbaca2fc165d5b7a P f26a9f62d2324ece8439899a2fa1236a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:51:43.266942 29788 tablet_bootstrap.cc:492] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db: Bootstrap starting.
I20260619 02:51:43.267329 29787 log.cc:824] T 4482dcfa907546b9bbaca2fc165d5b7a P c7002d3773974376bc5650372ef558b4: Log is configured to *not* fsync() on all Append() calls
I20260619 02:51:43.269824 29786 tablet_bootstrap.cc:492] T 4482dcfa907546b9bbaca2fc165d5b7a P f26a9f62d2324ece8439899a2fa1236a: No bootstrap required, opened a new log
I20260619 02:51:43.270064 29786 ts_tablet_manager.cc:1397] T 4482dcfa907546b9bbaca2fc165d5b7a P f26a9f62d2324ece8439899a2fa1236a: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.004s
I20260619 02:51:43.270237 29787 tablet_bootstrap.cc:492] T 4482dcfa907546b9bbaca2fc165d5b7a P c7002d3773974376bc5650372ef558b4: No bootstrap required, opened a new log
I20260619 02:51:43.270324 29788 tablet_bootstrap.cc:654] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db: Neither blocks nor log segments found. Creating new log.
I20260619 02:51:43.270486 29787 ts_tablet_manager.cc:1397] T 4482dcfa907546b9bbaca2fc165d5b7a P c7002d3773974376bc5650372ef558b4: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260619 02:51:43.271517 29788 log.cc:824] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db: Log is configured to *not* fsync() on all Append() calls
I20260619 02:51:43.274935 29788 tablet_bootstrap.cc:492] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db: No bootstrap required, opened a new log
I20260619 02:51:43.275194 29788 ts_tablet_manager.cc:1397] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260619 02:51:43.278039 29786 raft_consensus.cc:348] T 4482dcfa907546b9bbaca2fc165d5b7a P f26a9f62d2324ece8439899a2fa1236a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7002d3773974376bc5650372ef558b4" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 41551 } } peers { permanent_uuid: "d36667b8064843d9bc0f90a61276d4db" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36019 } } peers { permanent_uuid: "f26a9f62d2324ece8439899a2fa1236a" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 36899 } }
I20260619 02:51:43.279469 29786 raft_consensus.cc:374] T 4482dcfa907546b9bbaca2fc165d5b7a P f26a9f62d2324ece8439899a2fa1236a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:43.279593 29786 raft_consensus.cc:729] T 4482dcfa907546b9bbaca2fc165d5b7a P f26a9f62d2324ece8439899a2fa1236a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f26a9f62d2324ece8439899a2fa1236a, State: Initialized, Role: FOLLOWER
I20260619 02:51:43.280185 29786 consensus_queue.cc:260] T 4482dcfa907546b9bbaca2fc165d5b7a P f26a9f62d2324ece8439899a2fa1236a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7002d3773974376bc5650372ef558b4" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 41551 } } peers { permanent_uuid: "d36667b8064843d9bc0f90a61276d4db" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36019 } } peers { permanent_uuid: "f26a9f62d2324ece8439899a2fa1236a" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 36899 } }
I20260619 02:51:43.280465 29787 raft_consensus.cc:348] T 4482dcfa907546b9bbaca2fc165d5b7a P c7002d3773974376bc5650372ef558b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7002d3773974376bc5650372ef558b4" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 41551 } } peers { permanent_uuid: "d36667b8064843d9bc0f90a61276d4db" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36019 } } peers { permanent_uuid: "f26a9f62d2324ece8439899a2fa1236a" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 36899 } }
I20260619 02:51:43.280897 29787 raft_consensus.cc:374] T 4482dcfa907546b9bbaca2fc165d5b7a P c7002d3773974376bc5650372ef558b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:43.280987 29787 raft_consensus.cc:729] T 4482dcfa907546b9bbaca2fc165d5b7a P c7002d3773974376bc5650372ef558b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7002d3773974376bc5650372ef558b4, State: Initialized, Role: FOLLOWER
I20260619 02:51:43.281562 29787 consensus_queue.cc:260] T 4482dcfa907546b9bbaca2fc165d5b7a P c7002d3773974376bc5650372ef558b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7002d3773974376bc5650372ef558b4" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 41551 } } peers { permanent_uuid: "d36667b8064843d9bc0f90a61276d4db" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36019 } } peers { permanent_uuid: "f26a9f62d2324ece8439899a2fa1236a" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 36899 } }
I20260619 02:51:43.282723 29768 heartbeater.cc:503] Master 127.18.134.62:40705 was elected leader, sending a full tablet report...
I20260619 02:51:43.283324 29787 ts_tablet_manager.cc:1428] T 4482dcfa907546b9bbaca2fc165d5b7a P c7002d3773974376bc5650372ef558b4: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260619 02:51:43.284333 29635 heartbeater.cc:503] Master 127.18.134.62:40705 was elected leader, sending a full tablet report...
I20260619 02:51:43.284961 29786 ts_tablet_manager.cc:1428] T 4482dcfa907546b9bbaca2fc165d5b7a P f26a9f62d2324ece8439899a2fa1236a: Time spent starting tablet: real 0.015s user 0.007s sys 0.007s
I20260619 02:51:43.285282 29788 raft_consensus.cc:348] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7002d3773974376bc5650372ef558b4" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 41551 } } peers { permanent_uuid: "d36667b8064843d9bc0f90a61276d4db" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36019 } } peers { permanent_uuid: "f26a9f62d2324ece8439899a2fa1236a" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 36899 } }
I20260619 02:51:43.285706 29788 raft_consensus.cc:374] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:51:43.285795 29788 raft_consensus.cc:729] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d36667b8064843d9bc0f90a61276d4db, State: Initialized, Role: FOLLOWER
I20260619 02:51:43.286394 29788 consensus_queue.cc:260] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7002d3773974376bc5650372ef558b4" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 41551 } } peers { permanent_uuid: "d36667b8064843d9bc0f90a61276d4db" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36019 } } peers { permanent_uuid: "f26a9f62d2324ece8439899a2fa1236a" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 36899 } }
I20260619 02:51:43.288491 29788 ts_tablet_manager.cc:1428] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
W20260619 02:51:43.403975 29769 tablet.cc:2369] T 4482dcfa907546b9bbaca2fc165d5b7a P c7002d3773974376bc5650372ef558b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:51:43.429087 29636 tablet.cc:2369] T 4482dcfa907546b9bbaca2fc165d5b7a P f26a9f62d2324ece8439899a2fa1236a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:51:43.447126 29502 tablet.cc:2369] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:51:43.513172 29794 raft_consensus.cc:482] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:51:43.513476 29794 raft_consensus.cc:504] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7002d3773974376bc5650372ef558b4" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 41551 } } peers { permanent_uuid: "d36667b8064843d9bc0f90a61276d4db" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36019 } } peers { permanent_uuid: "f26a9f62d2324ece8439899a2fa1236a" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 36899 } }
I20260619 02:51:43.515213 29794 leader_election.cc:288] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7002d3773974376bc5650372ef558b4 (127.18.134.3:41551), f26a9f62d2324ece8439899a2fa1236a (127.18.134.2:36899)
I20260619 02:51:43.521898 29780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:43.515739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:58175 (local address 127.18.134.3:41551)
0619 02:51:43.516147 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:43.516153 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:43.516425 (+ 272us) server_negotiation.cc:407] Connection header received
0619 02:51:43.516626 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:43.516630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:43.516690 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:43.516777 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:43.517350 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.518101 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.520205 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.521025 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.521179 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:43.521255 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:43.521517 (+ 262us) server_negotiation.cc:299] Negotiation successful
0619 02:51:43.521646 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265}
I20260619 02:51:43.522038 29796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:43.515464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:41551 (local address 127.18.134.1:58175)
0619 02:51:43.516290 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:43.516311 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:43.516490 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:43.516811 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:43.516816 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:43.516842 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:43.517186 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:43.517195 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.518227 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.518232 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:43.520054 (+ 1822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:43.520067 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.521211 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.521217 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:43.521369 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:43.521393 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 02:51:43.521476 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 02:51:43.521627 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"mutex_wait_us":39,"thread_start_us":805,"threads_started":1}
I20260619 02:51:43.522336 29795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:43.515463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:36899 (local address 127.18.134.1:56029)
0619 02:51:43.516568 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0619 02:51:43.516589 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:51:43.516689 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:51:43.516990 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:51:43.516994 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:51:43.517020 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:51:43.517392 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:43.517401 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.518520 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.518523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:43.520310 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:51:43.520322 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.521550 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.521556 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:51:43.521701 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:43.521724 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 02:51:43.521803 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 02:51:43.521930 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":999,"mutex_wait_us":75,"thread_start_us":412,"threads_started":1}
I20260619 02:51:43.522215 29785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:43.515723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:56029 (local address 127.18.134.2:36899)
0619 02:51:43.515971 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:43.515978 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:43.516657 (+ 679us) server_negotiation.cc:407] Connection header received
0619 02:51:43.516811 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:43.516815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:43.516871 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:43.516960 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:51:43.517536 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.518362 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.520473 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.521345 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.521469 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:43.521539 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:43.521845 (+ 306us) server_negotiation.cc:299] Negotiation successful
0619 02:51:43.521978 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260619 02:51:43.523144 29723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4482dcfa907546b9bbaca2fc165d5b7a" candidate_uuid: "d36667b8064843d9bc0f90a61276d4db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7002d3773974376bc5650372ef558b4" is_pre_election: true
I20260619 02:51:43.523480 29590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4482dcfa907546b9bbaca2fc165d5b7a" candidate_uuid: "d36667b8064843d9bc0f90a61276d4db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f26a9f62d2324ece8439899a2fa1236a" is_pre_election: true
I20260619 02:51:43.523766 29723 raft_consensus.cc:2454] T 4482dcfa907546b9bbaca2fc165d5b7a P c7002d3773974376bc5650372ef558b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d36667b8064843d9bc0f90a61276d4db in term 0.
I20260619 02:51:43.523969 29590 raft_consensus.cc:2454] T 4482dcfa907546b9bbaca2fc165d5b7a P f26a9f62d2324ece8439899a2fa1236a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d36667b8064843d9bc0f90a61276d4db in term 0.
I20260619 02:51:43.524494 29392 leader_election.cc:302] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7002d3773974376bc5650372ef558b4, d36667b8064843d9bc0f90a61276d4db; no voters:
I20260619 02:51:43.524917 29794 raft_consensus.cc:2781] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:51:43.525063 29794 raft_consensus.cc:482] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:51:43.525159 29794 raft_consensus.cc:3037] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:43.527371 29794 raft_consensus.cc:504] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7002d3773974376bc5650372ef558b4" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 41551 } } peers { permanent_uuid: "d36667b8064843d9bc0f90a61276d4db" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36019 } } peers { permanent_uuid: "f26a9f62d2324ece8439899a2fa1236a" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 36899 } }
I20260619 02:51:43.527962 29794 leader_election.cc:288] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db [CANDIDATE]: Term 1 election: Requested vote from peers c7002d3773974376bc5650372ef558b4 (127.18.134.3:41551), f26a9f62d2324ece8439899a2fa1236a (127.18.134.2:36899)
I20260619 02:51:43.528419 29590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4482dcfa907546b9bbaca2fc165d5b7a" candidate_uuid: "d36667b8064843d9bc0f90a61276d4db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f26a9f62d2324ece8439899a2fa1236a"
I20260619 02:51:43.528596 29590 raft_consensus.cc:3037] T 4482dcfa907546b9bbaca2fc165d5b7a P f26a9f62d2324ece8439899a2fa1236a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:43.530730 29590 raft_consensus.cc:2454] T 4482dcfa907546b9bbaca2fc165d5b7a P f26a9f62d2324ece8439899a2fa1236a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d36667b8064843d9bc0f90a61276d4db in term 1.
I20260619 02:51:43.530866 29723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4482dcfa907546b9bbaca2fc165d5b7a" candidate_uuid: "d36667b8064843d9bc0f90a61276d4db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7002d3773974376bc5650372ef558b4"
I20260619 02:51:43.531082 29723 raft_consensus.cc:3037] T 4482dcfa907546b9bbaca2fc165d5b7a P c7002d3773974376bc5650372ef558b4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:51:43.531157 29389 leader_election.cc:302] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d36667b8064843d9bc0f90a61276d4db, f26a9f62d2324ece8439899a2fa1236a; no voters:
I20260619 02:51:43.531450 29794 raft_consensus.cc:2781] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:51:43.531939 29794 raft_consensus.cc:686] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db [term 1 LEADER]: Becoming Leader. State: Replica: d36667b8064843d9bc0f90a61276d4db, State: Running, Role: LEADER
I20260619 02:51:43.532532 29794 consensus_queue.cc:237] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7002d3773974376bc5650372ef558b4" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 41551 } } peers { permanent_uuid: "d36667b8064843d9bc0f90a61276d4db" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36019 } } peers { permanent_uuid: "f26a9f62d2324ece8439899a2fa1236a" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 36899 } }
I20260619 02:51:43.534211 29723 raft_consensus.cc:2454] T 4482dcfa907546b9bbaca2fc165d5b7a P c7002d3773974376bc5650372ef558b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d36667b8064843d9bc0f90a61276d4db in term 1.
I20260619 02:51:43.538077 29311 catalog_manager.cc:5495] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db reported cstate change: term changed from 0 to 1, leader changed from <none> to d36667b8064843d9bc0f90a61276d4db (127.18.134.1). New cstate: current_term: 1 leader_uuid: "d36667b8064843d9bc0f90a61276d4db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7002d3773974376bc5650372ef558b4" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 41551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d36667b8064843d9bc0f90a61276d4db" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36019 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f26a9f62d2324ece8439899a2fa1236a" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 36899 } health_report { overall_health: UNKNOWN } } }
I20260619 02:51:43.603492 29782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:43.598463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38498 (local address 127.18.134.1:36019)
0619 02:51:43.598696 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:43.598702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:43.598718 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:51:43.598812 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:43.598816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:43.598890 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:43.599022 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:51:43.599382 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.600165 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.601450 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.601849 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.601958 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:43.602422 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:51:43.602954 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:51:43.603092 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:43.603175 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 02:51:43.603253 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 02:51:43.611320 29590 raft_consensus.cc:1261] T 4482dcfa907546b9bbaca2fc165d5b7a P f26a9f62d2324ece8439899a2fa1236a [term 1 FOLLOWER]: Refusing update from remote peer d36667b8064843d9bc0f90a61276d4db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:51:43.611328 29723 raft_consensus.cc:1261] T 4482dcfa907546b9bbaca2fc165d5b7a P c7002d3773974376bc5650372ef558b4 [term 1 FOLLOWER]: Refusing update from remote peer d36667b8064843d9bc0f90a61276d4db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:51:43.612237 29794 consensus_queue.cc:1035] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db [LEADER]: Connected to new peer: Peer: permanent_uuid: "f26a9f62d2324ece8439899a2fa1236a" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 36899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:51:43.612552 29797 consensus_queue.cc:1035] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7002d3773974376bc5650372ef558b4" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 41551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:51:43.625214 29802 mvcc.cc:205] Tried to move back new op lower bound from 7298406414782472192 to 7298406414474313728. Current Snapshot: MvccSnapshot[applied={T|T < 7298406414782472192}]
I20260619 02:51:43.625530 29800 mvcc.cc:205] Tried to move back new op lower bound from 7298406414782472192 to 7298406414474313728. Current Snapshot: MvccSnapshot[applied={T|T < 7298406414782472192}]
I20260619 02:51:43.625964 29801 mvcc.cc:205] Tried to move back new op lower bound from 7298406414782472192 to 7298406414474313728. Current Snapshot: MvccSnapshot[applied={T|T < 7298406414782472192 or (T in {7298406414782472192})}]
I20260619 02:51:43.636559 29311 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:51:43.636806 29311 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:51:43.640275 29311 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c8da4dfb75443faac1678e18b79216d: Sending DeleteTablet for 3 replicas of tablet 4482dcfa907546b9bbaca2fc165d5b7a
I20260619 02:51:43.641414 29703 tablet_service.cc:1514] Processing DeleteTablet for tablet 4482dcfa907546b9bbaca2fc165d5b7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:51:43 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:50374
I20260619 02:51:43.642231 29809 tablet_replica.cc:330] T 4482dcfa907546b9bbaca2fc165d5b7a P c7002d3773974376bc5650372ef558b4: stopping tablet replica
I20260619 02:51:43.642915 29570 tablet_service.cc:1514] Processing DeleteTablet for tablet 4482dcfa907546b9bbaca2fc165d5b7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:51:43 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:57072
I20260619 02:51:43.643153 29809 raft_consensus.cc:2229] T 4482dcfa907546b9bbaca2fc165d5b7a P c7002d3773974376bc5650372ef558b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:51:43.643625 29809 raft_consensus.cc:2258] T 4482dcfa907546b9bbaca2fc165d5b7a P c7002d3773974376bc5650372ef558b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:51:43.643899 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 29373
I20260619 02:51:43.643927 29436 tablet_service.cc:1514] Processing DeleteTablet for tablet 4482dcfa907546b9bbaca2fc165d5b7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:51:43 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:38488
I20260619 02:51:43.645819 29809 ts_tablet_manager.cc:1905] T 4482dcfa907546b9bbaca2fc165d5b7a P c7002d3773974376bc5650372ef558b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:51:43.652995 29809 ts_tablet_manager.cc:1918] T 4482dcfa907546b9bbaca2fc165d5b7a P c7002d3773974376bc5650372ef558b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:51:43.653270 29809 log.cc:1196] T 4482dcfa907546b9bbaca2fc165d5b7a P c7002d3773974376bc5650372ef558b4: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-2/wal/wals/4482dcfa907546b9bbaca2fc165d5b7a
I20260619 02:51:43.653844 29809 ts_tablet_manager.cc:1939] T 4482dcfa907546b9bbaca2fc165d5b7a P c7002d3773974376bc5650372ef558b4: Deleting consensus metadata
I20260619 02:51:43.655202 29811 tablet_replica.cc:330] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db: stopping tablet replica
I20260619 02:51:43.655563 29811 raft_consensus.cc:2229] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:51:43.655923 29299 catalog_manager.cc:4841] TS c7002d3773974376bc5650372ef558b4 (127.18.134.3:41551): tablet 4482dcfa907546b9bbaca2fc165d5b7a (table test-table [id=bc2cdde458f248c38f7d08bd2a3d6f47]) successfully deleted
I20260619 02:51:43.656211 29811 raft_consensus.cc:2258] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:51:43.658073 29811 ts_tablet_manager.cc:1905] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:51:43.659158 29810 tablet_replica.cc:330] T 4482dcfa907546b9bbaca2fc165d5b7a P f26a9f62d2324ece8439899a2fa1236a: stopping tablet replica
I20260619 02:51:43.659919 29810 raft_consensus.cc:2229] T 4482dcfa907546b9bbaca2fc165d5b7a P f26a9f62d2324ece8439899a2fa1236a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:51:43.660416 29810 raft_consensus.cc:2258] T 4482dcfa907546b9bbaca2fc165d5b7a P f26a9f62d2324ece8439899a2fa1236a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:51:43.662710 29810 ts_tablet_manager.cc:1905] T 4482dcfa907546b9bbaca2fc165d5b7a P f26a9f62d2324ece8439899a2fa1236a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:51:43.665055 29811 ts_tablet_manager.cc:1918] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:51:43.665294 29811 log.cc:1196] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-0/wal/wals/4482dcfa907546b9bbaca2fc165d5b7a
I20260619 02:51:43.665848 29811 ts_tablet_manager.cc:1939] T 4482dcfa907546b9bbaca2fc165d5b7a P d36667b8064843d9bc0f90a61276d4db: Deleting consensus metadata
I20260619 02:51:43.667766 29296 catalog_manager.cc:4841] TS d36667b8064843d9bc0f90a61276d4db (127.18.134.1:36019): tablet 4482dcfa907546b9bbaca2fc165d5b7a (table test-table [id=bc2cdde458f248c38f7d08bd2a3d6f47]) successfully deleted
I20260619 02:51:43.668982 29810 ts_tablet_manager.cc:1918] T 4482dcfa907546b9bbaca2fc165d5b7a P f26a9f62d2324ece8439899a2fa1236a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:51:43.669121 29810 log.cc:1196] T 4482dcfa907546b9bbaca2fc165d5b7a P f26a9f62d2324ece8439899a2fa1236a: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781837410515220-18968-0/minicluster-data/ts-1/wal/wals/4482dcfa907546b9bbaca2fc165d5b7a
I20260619 02:51:43.669516 29810 ts_tablet_manager.cc:1939] T 4482dcfa907546b9bbaca2fc165d5b7a P f26a9f62d2324ece8439899a2fa1236a: Deleting consensus metadata
I20260619 02:51:43.670950 29296 catalog_manager.cc:4841] TS f26a9f62d2324ece8439899a2fa1236a (127.18.134.2:36899): tablet 4482dcfa907546b9bbaca2fc165d5b7a (table test-table [id=bc2cdde458f248c38f7d08bd2a3d6f47]) successfully deleted
Jun 19 02:51:43 dist-test-slave-9mr0 krb5kdc[29267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.1@KRBTEST.COM
I20260619 02:51:43.679242 29782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:43.645256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38510 (local address 127.18.134.1:36019)
0619 02:51:43.645419 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:43.645425 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:43.661071 (+ 15646us) server_negotiation.cc:407] Connection header received
0619 02:51:43.661368 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:43.661374 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:43.661456 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:43.661534 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:43.663327 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.664693 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.666532 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.667299 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.667400 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:43.672400 (+ 5000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:43.672410 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:43.672423 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:43.672467 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:43.676116 (+ 3649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:43.676965 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:43.676975 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:43.676979 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:43.677146 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:43.677550 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:43.677555 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:43.677559 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:43.678000 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:43.678284 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:43.678591 (+ 307us) server_negotiation.cc:299] Negotiation successful
0619 02:51:43.678874 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 02:51:43.679957 29496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:43.774534 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 29373
I20260619 02:51:43.788522 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 29507
Jun 19 02:51:43 dist-test-slave-9mr0 krb5kdc[29267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.2@KRBTEST.COM
I20260619 02:51:43.803602 29785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:43.789226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57076 (local address 127.18.134.2:36899)
0619 02:51:43.789388 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:43.789393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:43.789461 (+ 68us) server_negotiation.cc:407] Connection header received
0619 02:51:43.789633 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:43.789637 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:43.789696 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:43.789774 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:43.790968 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.791689 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.792931 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.793350 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.793455 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:43.797404 (+ 3949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:43.797419 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:43.797436 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:43.797479 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:43.800302 (+ 2823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:43.801018 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:43.801026 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:43.801029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:43.801175 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:43.801847 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:43.801853 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:43.801856 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:43.802352 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:43.802578 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:43.803058 (+ 480us) server_negotiation.cc:299] Negotiation successful
0619 02:51:43.803269 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 02:51:43.804451 29630 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:43.893422 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 29507
I20260619 02:51:43.907510 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 29640
Jun 19 02:51:43 dist-test-slave-9mr0 krb5kdc[29267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.3@KRBTEST.COM
I20260619 02:51:43.922997 29780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:51:43.908299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50382 (local address 127.18.134.3:41551)
0619 02:51:43.908491 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 02:51:43.908498 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:51:43.908596 (+ 98us) server_negotiation.cc:407] Connection header received
0619 02:51:43.908784 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:51:43.908790 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:51:43.908860 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:51:43.909071 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:51:43.910747 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.911498 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.912722 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:51:43.913122 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:51:43.913233 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:51:43.917570 (+ 4337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:51:43.917586 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:51:43.917602 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:51:43.917648 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:51:43.920308 (+ 2660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:43.920990 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:43.920998 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:43.921000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:43.921132 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:51:43.921493 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:51:43.921496 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:51:43.921498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:51:43.921866 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:51:43.922155 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0619 02:51:43.922438 (+ 283us) server_negotiation.cc:299] Negotiation successful
0619 02:51:43.922673 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 02:51:43.923635 29763 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:44.010154 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 29640
I20260619 02:51:44.025075 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 29280
I20260619 02:51:44.026219 29342 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:51:44.150084 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 29280
2026-06-19T02:51:44Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4195 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:51:44 dist-test-slave-9mr0 krb5kdc[29819](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:51:44 dist-test-slave-9mr0 krb5kdc[29819](info): set up 2 sockets
Jun 19 02:51:44 dist-test-slave-9mr0 krb5kdc[29819](info): commencing operation
krb5kdc: starting...
W20260619 02:51:45.175439 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.987s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:51:45 dist-test-slave-9mr0 krb5kdc[29819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:51:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:51:45Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.134.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.134.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.134.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/rangeradmin_127.18.134.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.134.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/rangeradmin_127.18.134.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.134.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.134.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.134.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/rangerlookup_127.18.134.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.134.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/rangerlookup_127.18.134.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.134.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.134.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.134.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/HTTP_127.18.134.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.134.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/HTTP_127.18.134.20@KRBTEST.COM.keytab.
I20260619 02:51:45.486043 18968 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskUJ45gP/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/postgres -l logfile start
2026-06-19 02:51:47.983 UTC [29849] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 02:51:47.983 UTC [29849] LOG: listening on IPv4 address "127.18.134.20", port 33449
2026-06-19 02:51:47.986 UTC [29849] LOG: listening on Unix socket "/tmp/.s.PGSQL.33449"
2026-06-19 02:51:48.008 UTC [29852] LOG: database system was shut down at 2026-06-19 02:51:46 UTC
2026-06-19 02:51:48.011 UTC [29849] LOG: database system is ready to accept connections
I20260619 02:51:48.914904 18968 mini_postgres.cc:96] Postgres bound to 33449
2026-06-19 02:51:48.931 UTC [29860] FATAL: database "slave" does not exist
127.18.134.20:33449 - accepting connections
I20260619 02:51:48.932657 18968 mini_ranger.cc:161] Starting Ranger...
I20260619 02:51:48.957669 18968 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 02:51:49.471664 18968 mini_ranger.cc:87] Created ranger Postgres database
I20260619 02:51:49.471802 18968 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ranger-admin
2026-06-19 02:51:49,753 [I] DB FLAVOR :POSTGRES
2026-06-19 02:51:49,754 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 02:51:49,754 [I] Checking connection..
2026-06-19 02:51:49,754 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:33449/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 02:51:50,012 [I] Checking connection passed.
2026-06-19 02:51:50,012 [I] --------- Verifying version history table ---------
2026-06-19 02:51:50,012 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:33449/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:51:50,272 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 02:51:50,272 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:33449/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:51:50,501 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 02:51:50,501 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 02:51:50,501 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:33449/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 02:51:50.757 UTC [29924] WARNING: there is no transaction in progress
2026-06-19 02:51:50,763 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 02:51:50,764 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:33449/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:51:51,021 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 02:51:51,022 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 02:51:51,022 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:33449/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 02:51:51,279 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:33449/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 02:51:51,535 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:33449/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-9mr0.c.gcp-upstream.internal','N') ;"
2026-06-19 02:51:51,772 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:33449/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 02:51:52,031 [I] Table x_portal_user does not exist in database ranger
2026-06-19 02:51:52,031 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:33449/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 02:51:52,288 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 02:51:52,289 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:33449/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 02:51:52,523 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:33449/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 02:51:52,767 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 02:51:52,768 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:33449/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 02:51:53.747 UTC [30032] WARNING: there is no transaction in progress
2026-06-19 02:51:53.763 UTC [30032] WARNING: there is no transaction in progress
2026-06-19 02:51:53.780 UTC [30032] WARNING: there is no transaction in progress
2026-06-19 02:51:53.797 UTC [30032] WARNING: there is no transaction in progress
2026-06-19 02:51:53.817 UTC [30032] WARNING: there is no transaction in progress
2026-06-19 02:51:53.839 UTC [30032] WARNING: there is no transaction in progress
2026-06-19 02:51:53.896 UTC [30032] WARNING: there is no transaction in progress
2026-06-19 02:51:53.903 UTC [30032] WARNING: there is no transaction in progress
2026-06-19 02:51:53.918 UTC [30032] WARNING: there is no transaction in progress
2026-06-19 02:51:53.929 UTC [30032] WARNING: there is no transaction in progress
2026-06-19 02:51:53.940 UTC [30032] WARNING: there is no transaction in progress
2026-06-19 02:51:53.955 UTC [30032] WARNING: there is no transaction in progress
2026-06-19 02:51:53.965 UTC [30032] WARNING: there is no transaction in progress
2026-06-19 02:51:53.972 UTC [30032] WARNING: there is no transaction in progress
2026-06-19 02:51:53.980 UTC [30032] WARNING: there is no transaction in progress
2026-06-19 02:51:53.987 UTC [30032] WARNING: there is no transaction in progress
2026-06-19 02:51:54,673 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 02:51:54,674 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:33449/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 02:51:54,922 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 02:51:54,922 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:33449/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 02:51:55,171 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 02:51:55,171 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:33449/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 02:51:55,419 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 02:51:55,419 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:33449/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 02:51:55,670 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:33449/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 02:51:55,900 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:33449/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-9mr0.c.gcp-upstream.internal';"
2026-06-19 02:51:56,150 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 02:51:56,150 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:33449/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 02:51:56,389 [I] DB_PATCHES have already been applied
I20260619 02:51:56.394928 18968 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ranger-admin:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskUJ45gP/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 02:51:56.395017 18968 mini_ranger.cc:193] Using host: 127.18.134.20
I20260619 02:51:56.408344 18968 mini_ranger.cc:227] Ranger admin URL: http://127.18.134.20:59823
Jun 19, 2026 2:51:56 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 2:51:56 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 2:51:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 2:51:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 2:51:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 2:51:57 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 2:51:57 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 02:51:57.434158 18968 mini_ranger.cc:160] Time spent starting Ranger: real 8.502s user 0.007s sys 0.033s
Jun 19, 2026 2:52:17 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 2:52:17 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 19, 2026 2:52:17 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 2:52:17 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 2:52:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 2:52:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 2:52:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 2:52:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 2:52:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 2:52:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 2:52:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 2:52:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 2:52:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 2:52:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 2:52:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 2:52:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 2:52:17 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 2:52:18 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 02:52:18.709375 18968 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.134.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.134.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.134.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/rangerkms_127.18.134.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.134.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/rangerkms_127.18.134.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.134.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.134.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.134.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/HTTP_127.18.134.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.134.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/HTTP_127.18.134.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 02:52:18 dist-test-slave-9mr0 krb5kdc[29819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837538, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 02:52:18.861359 18968 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 02:52:18.880952 18968 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 02:52:19.838212 18968 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 02:52:19.838389 18968 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ranger-kms
I20260619 02:52:19.838513 18968 mini_ranger_kms.cc:227] Using postgres at 127.18.134.20:33449
2026-06-19 02:52:20,228 [I] DB FLAVOR :POSTGRES
2026-06-19 02:52:20,228 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 02:52:20,228 [I] Checking connection
2026-06-19 02:52:20,228 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:33449/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 02:52:20,487 [I] connection success
2026-06-19 02:52:20,487 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 02:52:20,487 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 02:52:20,487 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:33449/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 02:52:20,714 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 02:52:20,715 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 02:52:20,715 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 02:52:20,715 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:33449/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 02:52:20,970 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 02:52:21.438795 18968 mini_ranger_kms.cc:310] Created kms service
I20260619 02:52:21.650174 18968 mini_ranger_kms.cc:326] Created kudu user
I20260619 02:52:21.697141 18968 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 02:52:21 dist-test-slave-9mr0 krb5kdc[29819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837538, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.134.20@KRBTEST.COM
I20260619 02:52:22.108850 18968 mini_ranger_kms.cc:384] Added ranger policy
I20260619 02:52:22.109161 18968 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ranger-kms:/tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 02:52:22.109220 18968 mini_ranger_kms.cc:240] Using host: 127.18.134.20
I20260619 02:52:22.128724 18968 mini_ranger_kms.cc:276] Ranger KMS PID: 30212
I20260619 02:52:22.128921 18968 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.134.20:37333
Jun 19, 2026 2:52:23 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 2:52:23 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 2:52:23 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 2:52:23 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 2:52:23 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 2:52:23 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 02:52:24.221764 18968 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.360s user 0.093s sys 0.318s
I20260619 02:52:24.222002 18968 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 02:52:24.222185 18968 mini_ranger_kms.cc:401] 127.18.134.20:37333/kms/v1/keys
Jun 19 02:52:24 dist-test-slave-9mr0 krb5kdc[29819](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781837544, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.134.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 02:52:24 dist-test-slave-9mr0 krb5kdc[29819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837544, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.134.20@KRBTEST.COM for HTTP/127.18.134.20@KRBTEST.COM
Jun 19 02:52:25 dist-test-slave-9mr0 krb5kdc[29819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837544, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.134.20@KRBTEST.COM for HTTP/127.18.134.20@KRBTEST.COM
Jun 19 02:52:27 dist-test-slave-9mr0 krb5kdc[29819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:52:27.618506 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:39747
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:36959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:39747
--ranger_config_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.134.20:37333/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:52:27.944777 30263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:52:27.945086 30263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:52:27.945205 30263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:52:27.952467 30263 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:52:27.952581 30263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:52:27.952651 30263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:52:27.952732 30263 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:52:27.952796 30263 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:52:27.963244 30263 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:36959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.134.20:37333/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:39747
--ranger_config_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:39747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:52:27.964524 30263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:52:27.965754 30263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:52:27.977385 30269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:52:27.977434 30268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:52:27.977437 30271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:52:27.978012 30263 server_base.cc:1034] running on GCE node
I20260619 02:52:27.978924 30263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:52:27.980785 30263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:52:27.981979 30263 hybrid_clock.cc:648] HybridClock initialized: now 1781837547981942 us; error 41 us; skew 500 ppm
I20260619 02:52:27.982306 30263 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:52:27 dist-test-slave-9mr0 krb5kdc[29819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837547, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:52:27.985371 30263 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:52:27.986943 30263 webserver.cc:460] Webserver started at http://127.18.134.62:45761/ using document root <none> and password file <none>
I20260619 02:52:27.987594 30263 fs_manager.cc:359] Metadata directory not provided
I20260619 02:52:27.987725 30263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:52:27.988010 30263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:52:27 dist-test-slave-9mr0 krb5kdc[29819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837547, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for HTTP/127.18.134.20@KRBTEST.COM
I20260619 02:52:28.037744 30263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "3828f5ddae5f45c7ac4d088b7c298ee4"
format_stamp: "Formatted at 2026-06-19 02:52:28 on dist-test-slave-9mr0"
server_key: "c964abf64c8444ecf8d589b212dd2d96"
server_key_iv: "c4c0e4792309686c0f51d4552c1c321f"
server_key_version: "kuduclusterkey@0"
I20260619 02:52:28.038682 30263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "3828f5ddae5f45c7ac4d088b7c298ee4"
format_stamp: "Formatted at 2026-06-19 02:52:28 on dist-test-slave-9mr0"
server_key: "c964abf64c8444ecf8d589b212dd2d96"
server_key_iv: "c4c0e4792309686c0f51d4552c1c321f"
server_key_version: "kuduclusterkey@0"
I20260619 02:52:28.044289 30263 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 02:52:28.063264 30278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:52:28.065122 30263 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260619 02:52:28.065317 30263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "3828f5ddae5f45c7ac4d088b7c298ee4"
format_stamp: "Formatted at 2026-06-19 02:52:28 on dist-test-slave-9mr0"
server_key: "c964abf64c8444ecf8d589b212dd2d96"
server_key_iv: "c4c0e4792309686c0f51d4552c1c321f"
server_key_version: "kuduclusterkey@0"
I20260619 02:52:28.065568 30263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:52:28.087939 30263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:52:28.093163 30263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:52:28.093511 30263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:52:28.113335 30263 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:39747
I20260619 02:52:28.113335 30329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:39747 every 8 connection(s)
I20260619 02:52:28.115043 30263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:52:28.119470 30330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:52:28.123083 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 30263
I20260619 02:52:28.123286 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:52:28.124038 18968 external_mini_cluster.cc:1428] Setting key e34e81dc66ae6ec6d2ffa39838f707bc
I20260619 02:52:28.133351 30330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3828f5ddae5f45c7ac4d088b7c298ee4: Bootstrap starting.
I20260619 02:52:28.137917 30330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3828f5ddae5f45c7ac4d088b7c298ee4: Neither blocks nor log segments found. Creating new log.
I20260619 02:52:28.140116 30330 log.cc:824] T 00000000000000000000000000000000 P 3828f5ddae5f45c7ac4d088b7c298ee4: Log is configured to *not* fsync() on all Append() calls
I20260619 02:52:28.144078 30330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3828f5ddae5f45c7ac4d088b7c298ee4: No bootstrap required, opened a new log
I20260619 02:52:28.153103 30330 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3828f5ddae5f45c7ac4d088b7c298ee4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3828f5ddae5f45c7ac4d088b7c298ee4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39747 } }
I20260619 02:52:28.153481 30330 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3828f5ddae5f45c7ac4d088b7c298ee4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:52:28.153614 30330 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3828f5ddae5f45c7ac4d088b7c298ee4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3828f5ddae5f45c7ac4d088b7c298ee4, State: Initialized, Role: FOLLOWER
I20260619 02:52:28.154353 30330 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3828f5ddae5f45c7ac4d088b7c298ee4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3828f5ddae5f45c7ac4d088b7c298ee4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39747 } }
Jun 19 02:52:28 dist-test-slave-9mr0 krb5kdc[29819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:52:28.154592 30330 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3828f5ddae5f45c7ac4d088b7c298ee4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:52:28.155170 30330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3828f5ddae5f45c7ac4d088b7c298ee4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:52:28.155346 30330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3828f5ddae5f45c7ac4d088b7c298ee4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:52:28.158516 30330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3828f5ddae5f45c7ac4d088b7c298ee4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3828f5ddae5f45c7ac4d088b7c298ee4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39747 } }
I20260619 02:52:28.159155 30330 leader_election.cc:302] T 00000000000000000000000000000000 P 3828f5ddae5f45c7ac4d088b7c298ee4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3828f5ddae5f45c7ac4d088b7c298ee4; no voters:
I20260619 02:52:28.159878 30330 leader_election.cc:288] T 00000000000000000000000000000000 P 3828f5ddae5f45c7ac4d088b7c298ee4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:52:28.159999 30335 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3828f5ddae5f45c7ac4d088b7c298ee4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:52:28.165896 30333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:52:28.126149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58632 (local address 127.18.134.62:39747)
0619 02:52:28.140846 (+ 14697us) server_negotiation.cc:206] Beginning negotiation
0619 02:52:28.140863 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 02:52:28.140923 (+ 60us) server_negotiation.cc:407] Connection header received
0619 02:52:28.143458 (+ 2535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:52:28.143490 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:52:28.143990 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:52:28.144438 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:52:28.145939 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:28.148026 (+ 2087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:28.149869 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:28.151017 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:28.151214 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:52:28.155778 (+ 4564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:52:28.155790 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:52:28.155812 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:52:28.155870 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:52:28.159693 (+ 3823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:52:28.160704 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:52:28.160716 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:52:28.160726 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:52:28.160933 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:52:28.161412 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:52:28.161417 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:52:28.161421 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:52:28.162199 (+ 778us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:52:28.162507 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0619 02:52:28.163907 (+ 1400us) server_negotiation.cc:299] Negotiation successful
0619 02:52:28.164397 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13999,"thread_start_us":13745,"threads_started":1}
I20260619 02:52:28.167321 30335 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3828f5ddae5f45c7ac4d088b7c298ee4 [term 1 LEADER]: Becoming Leader. State: Replica: 3828f5ddae5f45c7ac4d088b7c298ee4, State: Running, Role: LEADER
I20260619 02:52:28.168017 30335 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3828f5ddae5f45c7ac4d088b7c298ee4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3828f5ddae5f45c7ac4d088b7c298ee4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39747 } }
I20260619 02:52:28.170528 30336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3828f5ddae5f45c7ac4d088b7c298ee4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3828f5ddae5f45c7ac4d088b7c298ee4. Latest consensus state: current_term: 1 leader_uuid: "3828f5ddae5f45c7ac4d088b7c298ee4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3828f5ddae5f45c7ac4d088b7c298ee4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39747 } } }
I20260619 02:52:28.171025 30336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3828f5ddae5f45c7ac4d088b7c298ee4 [sys.catalog]: This master's current role is: LEADER
I20260619 02:52:28.171130 30330 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3828f5ddae5f45c7ac4d088b7c298ee4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:52:28.180112 30337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3828f5ddae5f45c7ac4d088b7c298ee4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3828f5ddae5f45c7ac4d088b7c298ee4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3828f5ddae5f45c7ac4d088b7c298ee4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39747 } } }
I20260619 02:52:28.180553 30337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3828f5ddae5f45c7ac4d088b7c298ee4 [sys.catalog]: This master's current role is: LEADER
I20260619 02:52:28.183255 30339 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:52:28.189121 30339 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:52:28.216763 30339 catalog_manager.cc:1347] Generated new cluster ID: 5c82f99c19704d3e801d81858d8209d8
I20260619 02:52:28.216897 30339 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:52:28.230329 30339 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:52:28.232210 30339 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:52:28.239944 30330 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 02:52:28.241179 30339 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3828f5ddae5f45c7ac4d088b7c298ee4: Generated new TSK 0
I20260619 02:52:28.242249 30339 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.134.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
I20260619 02:52:29.050959 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.134.20:37333/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:0
--local_ip_for_outbound_sockets=127.18.134.1
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:39747
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:36959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:52:29 dist-test-slave-9mr0 krb5kdc[29819](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781837549, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260619 02:52:29.487483 30376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:52:29.487875 30376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:52:29.488081 30376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:52:29.499616 30376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:52:29.499837 30376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:52:29.500056 30376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:52:29.516216 30376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:36959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.134.20:37333/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:39747
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:52:29.518123 30376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:52:29.519945 30376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:52:29.539291 30387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:52:29.547144 30386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:52:29.548018 30389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:52:29.550338 30376 server_base.cc:1034] running on GCE node
I20260619 02:52:29.551003 30376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:52:29.552006 30376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:52:29.553228 30376 hybrid_clock.cc:648] HybridClock initialized: now 1781837549553157 us; error 64 us; skew 500 ppm
I20260619 02:52:29.553548 30376 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:52:29 dist-test-slave-9mr0 krb5kdc[29819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837549, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:52:29.560915 30376 init.cc:377] Logged in from keytab as kudu/127.18.134.1@KRBTEST.COM (short username kudu)
I20260619 02:52:29.563076 30376 webserver.cc:460] Webserver started at http://127.18.134.1:36509/ using document root <none> and password file <none>
I20260619 02:52:29.564137 30376 fs_manager.cc:359] Metadata directory not provided
I20260619 02:52:29.564298 30376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:52:29.564669 30376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:52:29 dist-test-slave-9mr0 krb5kdc[29819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837549, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for HTTP/127.18.134.20@KRBTEST.COM
I20260619 02:52:29.589552 30376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-0/data/instance:
uuid: "f7d3f2bead4f4580b3aecbf55af61826"
format_stamp: "Formatted at 2026-06-19 02:52:29 on dist-test-slave-9mr0"
server_key: "ba52dd1428a5ad0909499463803a2646"
server_key_iv: "f8db4b2d5ca09d39a764ca6e43e9c326"
server_key_version: "kuduclusterkey@0"
I20260619 02:52:29.590605 30376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance:
uuid: "f7d3f2bead4f4580b3aecbf55af61826"
format_stamp: "Formatted at 2026-06-19 02:52:29 on dist-test-slave-9mr0"
server_key: "ba52dd1428a5ad0909499463803a2646"
server_key_iv: "f8db4b2d5ca09d39a764ca6e43e9c326"
server_key_version: "kuduclusterkey@0"
I20260619 02:52:29.617007 30376 fs_manager.cc:691] Time spent creating directory manager: real 0.026s user 0.008s sys 0.016s
I20260619 02:52:29.649153 30396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:52:29.652086 30376 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.002s sys 0.012s
I20260619 02:52:29.652372 30376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "f7d3f2bead4f4580b3aecbf55af61826"
format_stamp: "Formatted at 2026-06-19 02:52:29 on dist-test-slave-9mr0"
server_key: "ba52dd1428a5ad0909499463803a2646"
server_key_iv: "f8db4b2d5ca09d39a764ca6e43e9c326"
server_key_version: "kuduclusterkey@0"
I20260619 02:52:29.652750 30376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:52:29.705835 30376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:52:29.710256 30376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:52:29.710549 30376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:52:29.711609 30376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:52:29.713119 30376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:52:29.713261 30376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260619 02:52:29.713470 30376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:52:29.713590 30376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:52:29 dist-test-slave-9mr0 krb5kdc[29819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837549, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for HTTP/127.18.134.20@KRBTEST.COM
I20260619 02:52:29.870824 30376 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:39517
I20260619 02:52:29.870903 30508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:39517 every 8 connection(s)
I20260619 02:52:29.873196 30376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:52:29.882395 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 30376
I20260619 02:52:29.882606 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance
I20260619 02:52:29.883571 18968 external_mini_cluster.cc:1428] Setting key 9078f73e028f87232363be49aa100c6c
Jun 19 02:52:29 dist-test-slave-9mr0 krb5kdc[29819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837549, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:52:29.912259 30512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:52:29.878225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39747 (local address 127.18.134.1:57959)
0619 02:52:29.888139 (+ 9914us) negotiation.cc:107] Waiting for socket to connect
0619 02:52:29.888204 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0619 02:52:29.890599 (+ 2395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:52:29.892332 (+ 1733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:52:29.892346 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:52:29.892975 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:52:29.893784 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:52:29.893812 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:52:29.895467 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:52:29.895473 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:52:29.897891 (+ 2418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:52:29.897912 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:52:29.899163 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:52:29.899170 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:52:29.899435 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:52:29.900235 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:52:29.900264 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:52:29.903165 (+ 2901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:52:29.907110 (+ 3945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:52:29.907121 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:52:29.907141 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:52:29.907735 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:52:29.908166 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:52:29.908171 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:52:29.908175 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:52:29.908417 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:52:29.909565 (+ 1148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:52:29.909573 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:52:29.910026 (+ 453us) client_negotiation.cc:769] Sending connection context
0619 02:52:29.910322 (+ 296us) client_negotiation.cc:240] Negotiation successful
0619 02:52:29.910902 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9556,"thread_start_us":16951,"threads_started":1}
I20260619 02:52:29.912781 30511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:52:29.877506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:57959 (local address 127.18.134.62:39747)
0619 02:52:29.887533 (+ 10027us) server_negotiation.cc:206] Beginning negotiation
0619 02:52:29.887540 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:52:29.891100 (+ 3560us) server_negotiation.cc:407] Connection header received
0619 02:52:29.891171 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:52:29.891176 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:52:29.891268 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:52:29.891363 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:52:29.893994 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:29.895199 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:29.898071 (+ 2872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:29.898723 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:29.898851 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:52:29.903451 (+ 4600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:52:29.903458 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:52:29.903462 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:52:29.903504 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:52:29.906901 (+ 3397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:52:29.907882 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:52:29.907888 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:52:29.907892 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:52:29.908044 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:52:29.908539 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:52:29.908544 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:52:29.908547 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:52:29.909333 (+ 786us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:52:29.909472 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 02:52:29.912202 (+ 2730us) server_negotiation.cc:299] Negotiation successful
0619 02:52:29.912437 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9891,"thread_start_us":9764,"threads_started":1}
I20260619 02:52:29.914319 30509 heartbeater.cc:347] Connected to a master server at 127.18.134.62:39747
I20260619 02:52:29.914665 30509 heartbeater.cc:464] Registering TS with master...
I20260619 02:52:29.915493 30509 heartbeater.cc:511] Master 127.18.134.62:39747 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.134.2@KRBTEST.COM; defaulting to no policy
I20260619 02:52:29.918107 30295 ts_manager.cc:194] Registered new tserver with Master: f7d3f2bead4f4580b3aecbf55af61826 (127.18.134.1:39517)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.2@KRBTEST.COM" created.
I20260619 02:52:29.920403 30295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.1@KRBTEST.COM'} at 127.18.134.1:57959
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
I20260619 02:52:29.946439 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.134.20:37333/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:0
--local_ip_for_outbound_sockets=127.18.134.2
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:39747
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:36959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:52:30 dist-test-slave-9mr0 krb5kdc[29819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837549, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for HTTP/127.18.134.20@KRBTEST.COM
W20260619 02:52:30.422474 30515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:52:30.423010 30515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:52:30.423225 30515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:52:30.435585 30515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:52:30.435839 30515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:52:30.436074 30515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:52:30.451660 30515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:36959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.134.20:37333/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:39747
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:52:30.453706 30515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:52:30.455560 30515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:52:30.499163 30523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:52:30.503350 30526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:52:30.503773 30515 server_base.cc:1034] running on GCE node
W20260619 02:52:30.501312 30522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:52:30.505090 30515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:52:30.512648 30515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:52:30.514709 30515 hybrid_clock.cc:648] HybridClock initialized: now 1781837550514591 us; error 132 us; skew 500 ppm
I20260619 02:52:30.515239 30515 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:52:30 dist-test-slave-9mr0 krb5kdc[29819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837550, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:52:30.520668 30515 init.cc:377] Logged in from keytab as kudu/127.18.134.2@KRBTEST.COM (short username kudu)
I20260619 02:52:30.563092 30515 webserver.cc:460] Webserver started at http://127.18.134.2:38573/ using document root <none> and password file <none>
I20260619 02:52:30.564131 30515 fs_manager.cc:359] Metadata directory not provided
I20260619 02:52:30.564298 30515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:52:30.564663 30515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:52:30 dist-test-slave-9mr0 krb5kdc[29819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837550, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for HTTP/127.18.134.20@KRBTEST.COM
I20260619 02:52:30.584053 30515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-1/data/instance:
uuid: "ccd67b105ed04464b69e537a96d8722d"
format_stamp: "Formatted at 2026-06-19 02:52:30 on dist-test-slave-9mr0"
server_key: "bfbf70eda3306735085372053031f680"
server_key_iv: "e54b124728f7847909ffd95f563178a3"
server_key_version: "kuduclusterkey@0"
I20260619 02:52:30.585224 30515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance:
uuid: "ccd67b105ed04464b69e537a96d8722d"
format_stamp: "Formatted at 2026-06-19 02:52:30 on dist-test-slave-9mr0"
server_key: "bfbf70eda3306735085372053031f680"
server_key_iv: "e54b124728f7847909ffd95f563178a3"
server_key_version: "kuduclusterkey@0"
I20260619 02:52:30.606163 30515 fs_manager.cc:691] Time spent creating directory manager: real 0.020s user 0.003s sys 0.015s
I20260619 02:52:30.639267 30539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:52:30.641208 30515 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.003s sys 0.010s
I20260619 02:52:30.641491 30515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "ccd67b105ed04464b69e537a96d8722d"
format_stamp: "Formatted at 2026-06-19 02:52:30 on dist-test-slave-9mr0"
server_key: "bfbf70eda3306735085372053031f680"
server_key_iv: "e54b124728f7847909ffd95f563178a3"
server_key_version: "kuduclusterkey@0"
I20260619 02:52:30.641830 30515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:52:30.713424 30515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:52:30.718811 30515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:52:30.719208 30515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:52:30.729869 30515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:52:30.739349 30515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:52:30.739539 30515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:52:30.739727 30515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:52:30.739821 30515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:52:30.927484 30509 heartbeater.cc:503] Master 127.18.134.62:39747 was elected leader, sending a full tablet report...
I20260619 02:52:30.986675 30515 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:43969
I20260619 02:52:30.986737 30651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:43969 every 8 connection(s)
I20260619 02:52:30.988554 30515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:52:30.994988 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 30515
I20260619 02:52:30.995136 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance
I20260619 02:52:30.995630 18968 external_mini_cluster.cc:1428] Setting key 95955ac7891a4d1f2279582f1a1bdcaa
Jun 19 02:52:31 dist-test-slave-9mr0 krb5kdc[29819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837550, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.134.3@KRBTEST.COM; defaulting to no policy
I20260619 02:52:31.025816 30654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:52:30.991472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:35355 (local address 127.18.134.62:39747)
0619 02:52:31.003571 (+ 12099us) server_negotiation.cc:206] Beginning negotiation
0619 02:52:31.003577 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:52:31.007746 (+ 4169us) server_negotiation.cc:407] Connection header received
0619 02:52:31.008048 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:52:31.008054 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:52:31.008156 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:52:31.008255 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:52:31.010273 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.011357 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.013562 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.014185 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.014281 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:52:31.018752 (+ 4471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:52:31.018759 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:52:31.018764 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:52:31.018805 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:52:31.022222 (+ 3417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:52:31.023222 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:52:31.023229 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:52:31.023233 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:52:31.023385 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:52:31.023822 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:52:31.023828 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:52:31.023832 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:52:31.024150 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:52:31.024281 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0619 02:52:31.025047 (+ 766us) server_negotiation.cc:299] Negotiation successful
0619 02:52:31.025426 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11956,"thread_start_us":11821,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.3@KRBTEST.COM" created.
I20260619 02:52:31.030467 30655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:52:30.992184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39747 (local address 127.18.134.2:35355)
0619 02:52:31.004157 (+ 11973us) negotiation.cc:107] Waiting for socket to connect
0619 02:52:31.004216 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0619 02:52:31.006419 (+ 2203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:52:31.008699 (+ 2280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:52:31.008712 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:52:31.009273 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:52:31.010069 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:52:31.010094 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.011560 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.011565 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:52:31.013378 (+ 1813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:52:31.013393 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.014620 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.014649 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:52:31.014939 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:52:31.015716 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:52:31.015745 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:52:31.018425 (+ 2680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:52:31.022439 (+ 4014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:52:31.022448 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:52:31.022468 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:52:31.023075 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:52:31.023508 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:52:31.023512 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:52:31.023514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:52:31.023696 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:52:31.024337 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:52:31.024343 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:52:31.024683 (+ 340us) client_negotiation.cc:769] Sending connection context
0619 02:52:31.024930 (+ 247us) client_negotiation.cc:240] Negotiation successful
0619 02:52:31.025466 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11633,"thread_start_us":14344,"threads_started":1}
I20260619 02:52:31.032265 30652 heartbeater.cc:347] Connected to a master server at 127.18.134.62:39747
I20260619 02:52:31.032562 30652 heartbeater.cc:464] Registering TS with master...
I20260619 02:52:31.033363 30652 heartbeater.cc:511] Master 127.18.134.62:39747 requested a full tablet report, sending...
I20260619 02:52:31.034965 30295 ts_manager.cc:194] Registered new tserver with Master: ccd67b105ed04464b69e537a96d8722d (127.18.134.2:43969)
I20260619 02:52:31.035981 30295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.2@KRBTEST.COM'} at 127.18.134.2:35355
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
I20260619 02:52:31.055547 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.134.20:37333/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:0
--local_ip_for_outbound_sockets=127.18.134.3
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:39747
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:36959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:52:31.382562 30658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:52:31.382865 30658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:52:31.382980 30658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:52:31.390232 30658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:52:31.390345 30658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:52:31.390492 30658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:52:31.400559 30658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:36959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.134.20:37333/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:39747
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:52:31.401870 30658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:52:31.403155 30658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:52:31.416172 30664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:52:31.416446 30663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:52:31.416525 30666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:52:31.417416 30658 server_base.cc:1034] running on GCE node
I20260619 02:52:31.418037 30658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:52:31.419087 30658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:52:31.420264 30658 hybrid_clock.cc:648] HybridClock initialized: now 1781837551420202 us; error 55 us; skew 500 ppm
I20260619 02:52:31.420629 30658 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:52:31 dist-test-slave-9mr0 krb5kdc[29819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837551, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:52:31.423840 30658 init.cc:377] Logged in from keytab as kudu/127.18.134.3@KRBTEST.COM (short username kudu)
I20260619 02:52:31.425381 30658 webserver.cc:460] Webserver started at http://127.18.134.3:44213/ using document root <none> and password file <none>
I20260619 02:52:31.426086 30658 fs_manager.cc:359] Metadata directory not provided
I20260619 02:52:31.426196 30658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:52:31.426481 30658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:52:31 dist-test-slave-9mr0 krb5kdc[29819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837551, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for HTTP/127.18.134.20@KRBTEST.COM
I20260619 02:52:31.443048 30658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-2/data/instance:
uuid: "849a70d2d884463aa149794ef3dee846"
format_stamp: "Formatted at 2026-06-19 02:52:31 on dist-test-slave-9mr0"
server_key: "fc421060f2e6a05c6d599f18ecf21b8c"
server_key_iv: "e31be4dabae024c293c9fd0d78185a53"
server_key_version: "kuduclusterkey@0"
I20260619 02:52:31.443905 30658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance:
uuid: "849a70d2d884463aa149794ef3dee846"
format_stamp: "Formatted at 2026-06-19 02:52:31 on dist-test-slave-9mr0"
server_key: "fc421060f2e6a05c6d599f18ecf21b8c"
server_key_iv: "e31be4dabae024c293c9fd0d78185a53"
server_key_version: "kuduclusterkey@0"
I20260619 02:52:31.449376 30658 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260619 02:52:31.467672 30673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:52:31.469226 30658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:52:31.469403 30658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "849a70d2d884463aa149794ef3dee846"
format_stamp: "Formatted at 2026-06-19 02:52:31 on dist-test-slave-9mr0"
server_key: "fc421060f2e6a05c6d599f18ecf21b8c"
server_key_iv: "e31be4dabae024c293c9fd0d78185a53"
server_key_version: "kuduclusterkey@0"
I20260619 02:52:31.469658 30658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:52:31.500169 30658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:52:31.503777 30658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:52:31.504046 30658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:52:31.504957 30658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:52:31.506801 30658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:52:31.506875 30658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:52:31.506969 30658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:52:31.507014 30658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:52:31.539250 30658 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:44899
I20260619 02:52:31.539296 30785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:44899 every 8 connection(s)
I20260619 02:52:31.540465 30658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:52:31.549830 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 30658
I20260619 02:52:31.549996 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance
I20260619 02:52:31.550419 18968 external_mini_cluster.cc:1428] Setting key d6683a4ad8cc8a764773b532c6d831a6
Jun 19 02:52:31 dist-test-slave-9mr0 krb5kdc[29819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837551, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:52:31.560935 30788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:52:31.542948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:50051 (local address 127.18.134.62:39747)
0619 02:52:31.543436 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0619 02:52:31.543441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:52:31.544402 (+ 961us) server_negotiation.cc:407] Connection header received
0619 02:52:31.546167 (+ 1765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:52:31.546172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:52:31.546244 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:52:31.546353 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:52:31.548260 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.549295 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.551093 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.551718 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.551837 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:52:31.555396 (+ 3559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:52:31.555402 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:52:31.555405 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:52:31.555439 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:52:31.557863 (+ 2424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:52:31.558663 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:52:31.558669 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:52:31.558673 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:52:31.558801 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:52:31.559230 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:52:31.559233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:52:31.559235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:52:31.559540 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:52:31.559667 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 02:52:31.560357 (+ 690us) server_negotiation.cc:299] Negotiation successful
0619 02:52:31.560584 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":236,"threads_started":1}
I20260619 02:52:31.561887 30789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:52:31.543465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:39747 (local address 127.18.134.3:50051)
0619 02:52:31.544219 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0619 02:52:31.544269 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 02:52:31.545905 (+ 1636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:52:31.546677 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:52:31.546691 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:52:31.547173 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:52:31.547808 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:52:31.547827 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.549473 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.549478 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:52:31.550920 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:52:31.550933 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.551980 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.551988 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:52:31.552260 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:52:31.552964 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:52:31.552988 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:52:31.555169 (+ 2181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:52:31.558002 (+ 2833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:52:31.558011 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:52:31.558031 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:52:31.558524 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:52:31.558902 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:52:31.558907 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:52:31.558911 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:52:31.559109 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:52:31.559655 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:52:31.559662 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:52:31.560008 (+ 346us) client_negotiation.cc:769] Sending connection context
0619 02:52:31.560249 (+ 241us) client_negotiation.cc:240] Negotiation successful
0619 02:52:31.560799 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":304,"threads_started":1}
I20260619 02:52:31.563369 30786 heartbeater.cc:347] Connected to a master server at 127.18.134.62:39747
I20260619 02:52:31.563709 30786 heartbeater.cc:464] Registering TS with master...
I20260619 02:52:31.564476 30786 heartbeater.cc:511] Master 127.18.134.62:39747 requested a full tablet report, sending...
I20260619 02:52:31.565814 30295 ts_manager.cc:194] Registered new tserver with Master: 849a70d2d884463aa149794ef3dee846 (127.18.134.3:44899)
I20260619 02:52:31.566608 30295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.3@KRBTEST.COM'} at 127.18.134.3:50051
I20260619 02:52:31.578207 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:52:31.595572 30788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:52:31.582618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44050 (local address 127.18.134.62:39747)
0619 02:52:31.582824 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 02:52:31.582830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:52:31.583183 (+ 353us) server_negotiation.cc:407] Connection header received
0619 02:52:31.583347 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:52:31.583353 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:52:31.583428 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:52:31.583507 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:52:31.585022 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.585789 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.587380 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.587853 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.587977 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:52:31.590427 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:52:31.590432 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:52:31.590435 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:52:31.590469 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:52:31.593041 (+ 2572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:52:31.593706 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:52:31.593712 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:52:31.593716 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:52:31.593837 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:52:31.594188 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:52:31.594193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:52:31.594195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:52:31.594468 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:52:31.594714 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0619 02:52:31.595042 (+ 328us) server_negotiation.cc:299] Negotiation successful
0619 02:52:31.595267 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 02:52:31.600204 30295 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:52:31.607366 30295 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:52:31.630263 30800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:52:31.619796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:39517 (local address 127.0.0.1:40714)
0619 02:52:31.620590 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0619 02:52:31.620629 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:52:31.620805 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:52:31.622104 (+ 1299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:52:31.622112 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:52:31.622333 (+ 221us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:52:31.623062 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:52:31.623082 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.625106 (+ 2024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.625111 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:52:31.627494 (+ 2383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:52:31.627509 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.629302 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.629308 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:52:31.629491 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:52:31.629540 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 02:52:31.629650 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 02:52:31.629877 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"mutex_wait_us":19,"thread_start_us":7131,"threads_started":1}
I20260619 02:52:31.630236 30801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:52:31.620640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40714 (local address 127.18.134.1:39517)
0619 02:52:31.621445 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0619 02:52:31.621453 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:52:31.621483 (+ 30us) server_negotiation.cc:407] Connection header received
0619 02:52:31.621579 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:52:31.621586 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:52:31.621844 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:52:31.622206 (+ 362us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:52:31.623283 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.624816 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.627681 (+ 2865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.629116 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.629240 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:52:31.629482 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0619 02:52:31.629709 (+ 227us) server_negotiation.cc:299] Negotiation successful
0619 02:52:31.629876 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":486,"threads_started":1}
I20260619 02:52:31.631811 30802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:52:31.621053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44228 (local address 127.18.134.3:44899)
0619 02:52:31.621769 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0619 02:52:31.621776 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:52:31.623101 (+ 1325us) server_negotiation.cc:407] Connection header received
0619 02:52:31.623391 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:52:31.623399 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:52:31.623648 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:52:31.623806 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:52:31.624636 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.626124 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.629149 (+ 3025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.630712 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.630875 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:52:31.631100 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0619 02:52:31.631294 (+ 194us) server_negotiation.cc:299] Negotiation successful
0619 02:52:31.631453 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":730,"threads_started":1}
I20260619 02:52:31.631867 30803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:52:31.619819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:44899 (local address 127.0.0.1:44228)
0619 02:52:31.622980 (+ 3161us) negotiation.cc:107] Waiting for socket to connect
0619 02:52:31.623003 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 02:52:31.623210 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:52:31.624079 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:52:31.624084 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:52:31.624107 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:52:31.624463 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:52:31.624473 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.626329 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.626334 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:52:31.628986 (+ 2652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:52:31.629001 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.630942 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.630948 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:52:31.631106 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:52:31.631136 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 02:52:31.631220 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 02:52:31.631348 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3039,"mutex_wait_us":1,"thread_start_us":1471,"threads_started":1}
I20260619 02:52:31.635052 30444 tablet_service.cc:1467] Processing CreateTablet for tablet e593db9af02642c5be4500d97b0b2562 (DEFAULT_TABLE table=test-table [id=602263da4516454fa0ef30414cb86bf8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:52:31.636050 30799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:52:31.622436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:43969 (local address 127.0.0.1:57798)
0619 02:52:31.627218 (+ 4782us) negotiation.cc:107] Waiting for socket to connect
0619 02:52:31.627236 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:52:31.627336 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:52:31.628495 (+ 1159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:52:31.628500 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:52:31.628523 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:52:31.628903 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:52:31.628920 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.632407 (+ 3487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.632411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:52:31.633848 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:52:31.633861 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.635239 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.635245 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:52:31.635473 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:52:31.635498 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 02:52:31.635585 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 02:52:31.635726 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4656,"thread_start_us":2470,"threads_started":1}
I20260619 02:52:31.635591 30721 tablet_service.cc:1467] Processing CreateTablet for tablet e593db9af02642c5be4500d97b0b2562 (DEFAULT_TABLE table=test-table [id=602263da4516454fa0ef30414cb86bf8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:52:31.636117 30804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:52:31.622476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57798 (local address 127.18.134.2:43969)
0619 02:52:31.628004 (+ 5528us) server_negotiation.cc:206] Beginning negotiation
0619 02:52:31.628011 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:52:31.628034 (+ 23us) server_negotiation.cc:407] Connection header received
0619 02:52:31.628108 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:52:31.628113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:52:31.628305 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:52:31.628471 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:52:31.629073 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.630668 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.634005 (+ 3337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.635011 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.635138 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:52:31.635326 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0619 02:52:31.635641 (+ 315us) server_negotiation.cc:299] Negotiation successful
0619 02:52:31.635807 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5386,"thread_start_us":5279,"threads_started":1}
I20260619 02:52:31.636715 30444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e593db9af02642c5be4500d97b0b2562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:52:31.636876 30721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e593db9af02642c5be4500d97b0b2562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:52:31.639454 30587 tablet_service.cc:1467] Processing CreateTablet for tablet e593db9af02642c5be4500d97b0b2562 (DEFAULT_TABLE table=test-table [id=602263da4516454fa0ef30414cb86bf8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:52:31.640559 30587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e593db9af02642c5be4500d97b0b2562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:52:31.651970 30806 tablet_bootstrap.cc:492] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d: Bootstrap starting.
I20260619 02:52:31.655020 30807 tablet_bootstrap.cc:492] T e593db9af02642c5be4500d97b0b2562 P f7d3f2bead4f4580b3aecbf55af61826: Bootstrap starting.
I20260619 02:52:31.655349 30805 tablet_bootstrap.cc:492] T e593db9af02642c5be4500d97b0b2562 P 849a70d2d884463aa149794ef3dee846: Bootstrap starting.
I20260619 02:52:31.655809 30806 tablet_bootstrap.cc:654] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d: Neither blocks nor log segments found. Creating new log.
I20260619 02:52:31.657191 30806 log.cc:824] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d: Log is configured to *not* fsync() on all Append() calls
I20260619 02:52:31.659004 30807 tablet_bootstrap.cc:654] T e593db9af02642c5be4500d97b0b2562 P f7d3f2bead4f4580b3aecbf55af61826: Neither blocks nor log segments found. Creating new log.
I20260619 02:52:31.659653 30805 tablet_bootstrap.cc:654] T e593db9af02642c5be4500d97b0b2562 P 849a70d2d884463aa149794ef3dee846: Neither blocks nor log segments found. Creating new log.
I20260619 02:52:31.660287 30807 log.cc:824] T e593db9af02642c5be4500d97b0b2562 P f7d3f2bead4f4580b3aecbf55af61826: Log is configured to *not* fsync() on all Append() calls
I20260619 02:52:31.660713 30805 log.cc:824] T e593db9af02642c5be4500d97b0b2562 P 849a70d2d884463aa149794ef3dee846: Log is configured to *not* fsync() on all Append() calls
I20260619 02:52:31.660989 30806 tablet_bootstrap.cc:492] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d: No bootstrap required, opened a new log
I20260619 02:52:31.661245 30806 ts_tablet_manager.cc:1397] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.006s
I20260619 02:52:31.663360 30807 tablet_bootstrap.cc:492] T e593db9af02642c5be4500d97b0b2562 P f7d3f2bead4f4580b3aecbf55af61826: No bootstrap required, opened a new log
I20260619 02:52:31.663645 30807 ts_tablet_manager.cc:1397] T e593db9af02642c5be4500d97b0b2562 P f7d3f2bead4f4580b3aecbf55af61826: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260619 02:52:31.663800 30805 tablet_bootstrap.cc:492] T e593db9af02642c5be4500d97b0b2562 P 849a70d2d884463aa149794ef3dee846: No bootstrap required, opened a new log
I20260619 02:52:31.664060 30805 ts_tablet_manager.cc:1397] T e593db9af02642c5be4500d97b0b2562 P 849a70d2d884463aa149794ef3dee846: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260619 02:52:31.668403 30806 raft_consensus.cc:348] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d3f2bead4f4580b3aecbf55af61826" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 39517 } } peers { permanent_uuid: "849a70d2d884463aa149794ef3dee846" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44899 } } peers { permanent_uuid: "ccd67b105ed04464b69e537a96d8722d" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43969 } }
I20260619 02:52:31.668818 30806 raft_consensus.cc:374] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:52:31.668916 30806 raft_consensus.cc:729] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccd67b105ed04464b69e537a96d8722d, State: Initialized, Role: FOLLOWER
I20260619 02:52:31.669499 30806 consensus_queue.cc:260] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d3f2bead4f4580b3aecbf55af61826" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 39517 } } peers { permanent_uuid: "849a70d2d884463aa149794ef3dee846" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44899 } } peers { permanent_uuid: "ccd67b105ed04464b69e537a96d8722d" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43969 } }
I20260619 02:52:31.672367 30652 heartbeater.cc:503] Master 127.18.134.62:39747 was elected leader, sending a full tablet report...
I20260619 02:52:31.673262 30806 ts_tablet_manager.cc:1428] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260619 02:52:31.675273 30805 raft_consensus.cc:348] T e593db9af02642c5be4500d97b0b2562 P 849a70d2d884463aa149794ef3dee846 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d3f2bead4f4580b3aecbf55af61826" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 39517 } } peers { permanent_uuid: "849a70d2d884463aa149794ef3dee846" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44899 } } peers { permanent_uuid: "ccd67b105ed04464b69e537a96d8722d" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43969 } }
I20260619 02:52:31.675273 30807 raft_consensus.cc:348] T e593db9af02642c5be4500d97b0b2562 P f7d3f2bead4f4580b3aecbf55af61826 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d3f2bead4f4580b3aecbf55af61826" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 39517 } } peers { permanent_uuid: "849a70d2d884463aa149794ef3dee846" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44899 } } peers { permanent_uuid: "ccd67b105ed04464b69e537a96d8722d" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43969 } }
I20260619 02:52:31.675716 30805 raft_consensus.cc:374] T e593db9af02642c5be4500d97b0b2562 P 849a70d2d884463aa149794ef3dee846 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:52:31.675716 30807 raft_consensus.cc:374] T e593db9af02642c5be4500d97b0b2562 P f7d3f2bead4f4580b3aecbf55af61826 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:52:31.675820 30807 raft_consensus.cc:729] T e593db9af02642c5be4500d97b0b2562 P f7d3f2bead4f4580b3aecbf55af61826 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7d3f2bead4f4580b3aecbf55af61826, State: Initialized, Role: FOLLOWER
I20260619 02:52:31.675822 30805 raft_consensus.cc:729] T e593db9af02642c5be4500d97b0b2562 P 849a70d2d884463aa149794ef3dee846 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 849a70d2d884463aa149794ef3dee846, State: Initialized, Role: FOLLOWER
I20260619 02:52:31.676503 30807 consensus_queue.cc:260] T e593db9af02642c5be4500d97b0b2562 P f7d3f2bead4f4580b3aecbf55af61826 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d3f2bead4f4580b3aecbf55af61826" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 39517 } } peers { permanent_uuid: "849a70d2d884463aa149794ef3dee846" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44899 } } peers { permanent_uuid: "ccd67b105ed04464b69e537a96d8722d" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43969 } }
I20260619 02:52:31.676590 30805 consensus_queue.cc:260] T e593db9af02642c5be4500d97b0b2562 P 849a70d2d884463aa149794ef3dee846 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d3f2bead4f4580b3aecbf55af61826" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 39517 } } peers { permanent_uuid: "849a70d2d884463aa149794ef3dee846" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44899 } } peers { permanent_uuid: "ccd67b105ed04464b69e537a96d8722d" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43969 } }
I20260619 02:52:31.678071 30786 heartbeater.cc:503] Master 127.18.134.62:39747 was elected leader, sending a full tablet report...
I20260619 02:52:31.678717 30807 ts_tablet_manager.cc:1428] T e593db9af02642c5be4500d97b0b2562 P f7d3f2bead4f4580b3aecbf55af61826: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260619 02:52:31.679960 30805 ts_tablet_manager.cc:1428] T e593db9af02642c5be4500d97b0b2562 P 849a70d2d884463aa149794ef3dee846: Time spent starting tablet: real 0.016s user 0.007s sys 0.008s
W20260619 02:52:31.746845 30653 tablet.cc:2369] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:52:31.791910 30787 tablet.cc:2369] T e593db9af02642c5be4500d97b0b2562 P 849a70d2d884463aa149794ef3dee846: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:52:31.794996 30811 raft_consensus.cc:482] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:52:31.795270 30811 raft_consensus.cc:504] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d3f2bead4f4580b3aecbf55af61826" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 39517 } } peers { permanent_uuid: "849a70d2d884463aa149794ef3dee846" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44899 } } peers { permanent_uuid: "ccd67b105ed04464b69e537a96d8722d" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43969 } }
I20260619 02:52:31.796845 30811 leader_election.cc:288] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7d3f2bead4f4580b3aecbf55af61826 (127.18.134.1:39517), 849a70d2d884463aa149794ef3dee846 (127.18.134.3:44899)
I20260619 02:52:31.803524 30801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:52:31.797378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:44363 (local address 127.18.134.1:39517)
0619 02:52:31.797585 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 02:52:31.797591 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:52:31.798570 (+ 979us) server_negotiation.cc:407] Connection header received
0619 02:52:31.798804 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:52:31.798808 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:52:31.798874 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:52:31.798947 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:52:31.799423 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.800162 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.801858 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.802653 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.802811 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:52:31.802889 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:52:31.803140 (+ 251us) server_negotiation.cc:299] Negotiation successful
0619 02:52:31.803271 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 02:52:31.803828 30814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:52:31.797286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:39517 (local address 127.18.134.2:44363)
0619 02:52:31.798441 (+ 1155us) negotiation.cc:107] Waiting for socket to connect
0619 02:52:31.798460 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:52:31.798655 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:52:31.798978 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:52:31.798983 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:52:31.799013 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:52:31.799291 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:52:31.799300 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.800284 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.800289 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:52:31.801704 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:52:31.801715 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.802851 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.802857 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:52:31.802989 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:52:31.803015 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 02:52:31.803118 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 02:52:31.803269 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1052,"thread_start_us":875,"threads_started":1}
I20260619 02:52:31.803913 30802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:52:31.797365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:33889 (local address 127.18.134.3:44899)
0619 02:52:31.797591 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0619 02:52:31.797596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:52:31.798993 (+ 1397us) server_negotiation.cc:407] Connection header received
0619 02:52:31.799186 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:52:31.799190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:52:31.799247 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:52:31.799319 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:52:31.799810 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.800516 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.802111 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.802915 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.803059 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:52:31.803140 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 02:52:31.803555 (+ 415us) server_negotiation.cc:299] Negotiation successful
0619 02:52:31.803683 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260619 02:52:31.804124 30815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:52:31.797281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:44899 (local address 127.18.134.2:33889)
0619 02:52:31.798885 (+ 1604us) negotiation.cc:107] Waiting for socket to connect
0619 02:52:31.798900 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:52:31.799058 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:52:31.799343 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:52:31.799347 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:52:31.799368 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:52:31.799677 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:52:31.799685 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.800640 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.800646 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:52:31.801968 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:52:31.801979 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.803106 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.803112 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:52:31.803377 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:52:31.803407 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 02:52:31.803509 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 02:52:31.803680 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1484,"mutex_wait_us":34,"spinlock_wait_cycles":36096,"thread_start_us":1213,"threads_started":1}
I20260619 02:52:31.805009 30741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e593db9af02642c5be4500d97b0b2562" candidate_uuid: "ccd67b105ed04464b69e537a96d8722d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "849a70d2d884463aa149794ef3dee846" is_pre_election: true
I20260619 02:52:31.805022 30464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e593db9af02642c5be4500d97b0b2562" candidate_uuid: "ccd67b105ed04464b69e537a96d8722d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7d3f2bead4f4580b3aecbf55af61826" is_pre_election: true
I20260619 02:52:31.805574 30464 raft_consensus.cc:2454] T e593db9af02642c5be4500d97b0b2562 P f7d3f2bead4f4580b3aecbf55af61826 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccd67b105ed04464b69e537a96d8722d in term 0.
I20260619 02:52:31.805575 30741 raft_consensus.cc:2454] T e593db9af02642c5be4500d97b0b2562 P 849a70d2d884463aa149794ef3dee846 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccd67b105ed04464b69e537a96d8722d in term 0.
I20260619 02:52:31.806265 30543 leader_election.cc:302] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccd67b105ed04464b69e537a96d8722d, f7d3f2bead4f4580b3aecbf55af61826; no voters:
I20260619 02:52:31.806622 30811 raft_consensus.cc:2781] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:52:31.806790 30811 raft_consensus.cc:482] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:52:31.806890 30811 raft_consensus.cc:3037] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:52:31.809450 30811 raft_consensus.cc:504] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d3f2bead4f4580b3aecbf55af61826" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 39517 } } peers { permanent_uuid: "849a70d2d884463aa149794ef3dee846" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44899 } } peers { permanent_uuid: "ccd67b105ed04464b69e537a96d8722d" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43969 } }
I20260619 02:52:31.810082 30811 leader_election.cc:288] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d [CANDIDATE]: Term 1 election: Requested vote from peers f7d3f2bead4f4580b3aecbf55af61826 (127.18.134.1:39517), 849a70d2d884463aa149794ef3dee846 (127.18.134.3:44899)
I20260619 02:52:31.810727 30464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e593db9af02642c5be4500d97b0b2562" candidate_uuid: "ccd67b105ed04464b69e537a96d8722d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7d3f2bead4f4580b3aecbf55af61826"
I20260619 02:52:31.810730 30741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e593db9af02642c5be4500d97b0b2562" candidate_uuid: "ccd67b105ed04464b69e537a96d8722d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "849a70d2d884463aa149794ef3dee846"
I20260619 02:52:31.810954 30741 raft_consensus.cc:3037] T e593db9af02642c5be4500d97b0b2562 P 849a70d2d884463aa149794ef3dee846 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:52:31.810954 30464 raft_consensus.cc:3037] T e593db9af02642c5be4500d97b0b2562 P f7d3f2bead4f4580b3aecbf55af61826 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:52:31.814293 30741 raft_consensus.cc:2454] T e593db9af02642c5be4500d97b0b2562 P 849a70d2d884463aa149794ef3dee846 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccd67b105ed04464b69e537a96d8722d in term 1.
I20260619 02:52:31.814301 30464 raft_consensus.cc:2454] T e593db9af02642c5be4500d97b0b2562 P f7d3f2bead4f4580b3aecbf55af61826 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccd67b105ed04464b69e537a96d8722d in term 1.
I20260619 02:52:31.814841 30543 leader_election.cc:302] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 849a70d2d884463aa149794ef3dee846, ccd67b105ed04464b69e537a96d8722d, f7d3f2bead4f4580b3aecbf55af61826; no voters:
I20260619 02:52:31.815207 30811 raft_consensus.cc:2781] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:52:31.815690 30811 raft_consensus.cc:686] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d [term 1 LEADER]: Becoming Leader. State: Replica: ccd67b105ed04464b69e537a96d8722d, State: Running, Role: LEADER
I20260619 02:52:31.816183 30811 consensus_queue.cc:237] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d3f2bead4f4580b3aecbf55af61826" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 39517 } } peers { permanent_uuid: "849a70d2d884463aa149794ef3dee846" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44899 } } peers { permanent_uuid: "ccd67b105ed04464b69e537a96d8722d" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43969 } }
I20260619 02:52:31.820958 30294 catalog_manager.cc:5495] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d reported cstate change: term changed from 0 to 1, leader changed from <none> to ccd67b105ed04464b69e537a96d8722d (127.18.134.2). New cstate: current_term: 1 leader_uuid: "ccd67b105ed04464b69e537a96d8722d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7d3f2bead4f4580b3aecbf55af61826" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 39517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "849a70d2d884463aa149794ef3dee846" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccd67b105ed04464b69e537a96d8722d" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43969 } health_report { overall_health: HEALTHY } } }
I20260619 02:52:31.869946 30804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:52:31.864590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57806 (local address 127.18.134.2:43969)
0619 02:52:31.864811 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 02:52:31.864816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:52:31.864852 (+ 36us) server_negotiation.cc:407] Connection header received
0619 02:52:31.865030 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:52:31.865034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:52:31.865124 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:52:31.865252 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:52:31.865552 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.866228 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.867655 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.868081 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.868184 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:52:31.868802 (+ 618us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:52:31.869365 (+ 563us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:52:31.869481 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 02:52:31.869598 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 02:52:31.869695 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 02:52:31.877861 30741 raft_consensus.cc:1261] T e593db9af02642c5be4500d97b0b2562 P 849a70d2d884463aa149794ef3dee846 [term 1 FOLLOWER]: Refusing update from remote peer ccd67b105ed04464b69e537a96d8722d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:52:31.877874 30464 raft_consensus.cc:1261] T e593db9af02642c5be4500d97b0b2562 P f7d3f2bead4f4580b3aecbf55af61826 [term 1 FOLLOWER]: Refusing update from remote peer ccd67b105ed04464b69e537a96d8722d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:52:31.878813 30816 consensus_queue.cc:1035] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7d3f2bead4f4580b3aecbf55af61826" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 39517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:52:31.879141 30811 consensus_queue.cc:1035] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d [LEADER]: Connected to new peer: Peer: permanent_uuid: "849a70d2d884463aa149794ef3dee846" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:52:31.892196 30821 mvcc.cc:205] Tried to move back new op lower bound from 7298406612482580480 to 7298406612242661376. Current Snapshot: MvccSnapshot[applied={T|T < 7298406612482580480}]
I20260619 02:52:31.911175 30294 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:52:31.911504 30294 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:52:31.915436 30294 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3828f5ddae5f45c7ac4d088b7c298ee4: Sending DeleteTablet for 3 replicas of tablet e593db9af02642c5be4500d97b0b2562
I20260619 02:52:31.916718 30721 tablet_service.cc:1514] Processing DeleteTablet for tablet e593db9af02642c5be4500d97b0b2562 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:52:31 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:44228
I20260619 02:52:31.916718 30444 tablet_service.cc:1514] Processing DeleteTablet for tablet e593db9af02642c5be4500d97b0b2562 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:52:31 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:40714
I20260619 02:52:31.917547 30587 tablet_service.cc:1514] Processing DeleteTablet for tablet e593db9af02642c5be4500d97b0b2562 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:52:31 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:57798
I20260619 02:52:31.919291 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 30376
I20260619 02:52:31.927385 30831 tablet_replica.cc:330] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d: stopping tablet replica
I20260619 02:52:31.927642 30830 tablet_replica.cc:330] T e593db9af02642c5be4500d97b0b2562 P f7d3f2bead4f4580b3aecbf55af61826: stopping tablet replica
I20260619 02:52:31.927932 30831 raft_consensus.cc:2229] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:52:31.928114 30830 raft_consensus.cc:2229] T e593db9af02642c5be4500d97b0b2562 P f7d3f2bead4f4580b3aecbf55af61826 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:52:31.928647 30830 raft_consensus.cc:2258] T e593db9af02642c5be4500d97b0b2562 P f7d3f2bead4f4580b3aecbf55af61826 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:52:31.928700 30831 raft_consensus.cc:2258] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:52:31.929466 30829 tablet_replica.cc:330] T e593db9af02642c5be4500d97b0b2562 P 849a70d2d884463aa149794ef3dee846: stopping tablet replica
I20260619 02:52:31.929898 30829 raft_consensus.cc:2229] T e593db9af02642c5be4500d97b0b2562 P 849a70d2d884463aa149794ef3dee846 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:52:31.930401 30829 raft_consensus.cc:2258] T e593db9af02642c5be4500d97b0b2562 P 849a70d2d884463aa149794ef3dee846 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:52:31.931008 30830 ts_tablet_manager.cc:1905] T e593db9af02642c5be4500d97b0b2562 P f7d3f2bead4f4580b3aecbf55af61826: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:52:31.932749 30829 ts_tablet_manager.cc:1905] T e593db9af02642c5be4500d97b0b2562 P 849a70d2d884463aa149794ef3dee846: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:52:31.938570 30830 ts_tablet_manager.cc:1918] T e593db9af02642c5be4500d97b0b2562 P f7d3f2bead4f4580b3aecbf55af61826: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:52:31.938778 30830 log.cc:1196] T e593db9af02642c5be4500d97b0b2562 P f7d3f2bead4f4580b3aecbf55af61826: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-0/wal/wals/e593db9af02642c5be4500d97b0b2562
I20260619 02:52:31.939293 30830 ts_tablet_manager.cc:1939] T e593db9af02642c5be4500d97b0b2562 P f7d3f2bead4f4580b3aecbf55af61826: Deleting consensus metadata
I20260619 02:52:31.939320 30829 ts_tablet_manager.cc:1918] T e593db9af02642c5be4500d97b0b2562 P 849a70d2d884463aa149794ef3dee846: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:52:31.939460 30829 log.cc:1196] T e593db9af02642c5be4500d97b0b2562 P 849a70d2d884463aa149794ef3dee846: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-2/wal/wals/e593db9af02642c5be4500d97b0b2562
Jun 19 02:52:31 dist-test-slave-9mr0 krb5kdc[29819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.1@KRBTEST.COM
I20260619 02:52:31.939918 30829 ts_tablet_manager.cc:1939] T e593db9af02642c5be4500d97b0b2562 P 849a70d2d884463aa149794ef3dee846: Deleting consensus metadata
I20260619 02:52:31.941192 30282 catalog_manager.cc:4841] TS f7d3f2bead4f4580b3aecbf55af61826 (127.18.134.1:39517): tablet e593db9af02642c5be4500d97b0b2562 (table test-table [id=602263da4516454fa0ef30414cb86bf8]) successfully deleted
I20260619 02:52:31.941507 30281 catalog_manager.cc:4841] TS 849a70d2d884463aa149794ef3dee846 (127.18.134.3:44899): tablet e593db9af02642c5be4500d97b0b2562 (table test-table [id=602263da4516454fa0ef30414cb86bf8]) successfully deleted
I20260619 02:52:31.943019 30831 ts_tablet_manager.cc:1905] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:52:31.948354 30801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:52:31.920508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40730 (local address 127.18.134.1:39517)
0619 02:52:31.920705 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 02:52:31.920713 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:52:31.929640 (+ 8927us) server_negotiation.cc:407] Connection header received
0619 02:52:31.929955 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:52:31.929961 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:52:31.930040 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:52:31.930114 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:52:31.932201 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.933268 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.935144 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:31.935798 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:31.935894 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:52:31.941221 (+ 5327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:52:31.941235 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:52:31.941252 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:52:31.941290 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:52:31.945009 (+ 3719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:52:31.945996 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:52:31.946005 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:52:31.946009 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:52:31.946161 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:52:31.946572 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:52:31.946577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:52:31.946580 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:52:31.947093 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:52:31.947384 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0619 02:52:31.947742 (+ 358us) server_negotiation.cc:299] Negotiation successful
0619 02:52:31.947980 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 02:52:31.949131 30504 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:52:31.950186 30831 ts_tablet_manager.cc:1918] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:52:31.950362 30831 log.cc:1196] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781837410515220-18968-0/minicluster-data/ts-1/wal/wals/e593db9af02642c5be4500d97b0b2562
I20260619 02:52:31.950888 30831 ts_tablet_manager.cc:1939] T e593db9af02642c5be4500d97b0b2562 P ccd67b105ed04464b69e537a96d8722d: Deleting consensus metadata
I20260619 02:52:31.952579 30281 catalog_manager.cc:4841] TS ccd67b105ed04464b69e537a96d8722d (127.18.134.2:43969): tablet e593db9af02642c5be4500d97b0b2562 (table test-table [id=602263da4516454fa0ef30414cb86bf8]) successfully deleted
I20260619 02:52:32.038542 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 30376
I20260619 02:52:32.053398 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 30515
Jun 19 02:52:32 dist-test-slave-9mr0 krb5kdc[29819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.2@KRBTEST.COM
I20260619 02:52:32.070328 30804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:52:32.054106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57808 (local address 127.18.134.2:43969)
0619 02:52:32.054292 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 02:52:32.054298 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:52:32.054319 (+ 21us) server_negotiation.cc:407] Connection header received
0619 02:52:32.054425 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:52:32.054430 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:52:32.054539 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:52:32.054680 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:52:32.055923 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:32.057104 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:32.058345 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:32.059047 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:32.059181 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:52:32.063470 (+ 4289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:52:32.063488 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:52:32.063510 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:52:32.063570 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:52:32.067415 (+ 3845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:52:32.068160 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:52:32.068169 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:52:32.068173 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:52:32.068351 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:52:32.068706 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:52:32.068711 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:52:32.068714 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:52:32.069180 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:52:32.069483 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0619 02:52:32.069710 (+ 227us) server_negotiation.cc:299] Negotiation successful
0619 02:52:32.069937 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 02:52:32.071105 30647 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:52:32.180943 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 30515
I20260619 02:52:32.194392 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 30658
Jun 19 02:52:32 dist-test-slave-9mr0 krb5kdc[29819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.3@KRBTEST.COM
I20260619 02:52:32.208876 30802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:52:32.195072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44244 (local address 127.18.134.3:44899)
0619 02:52:32.195233 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 02:52:32.195238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:52:32.195255 (+ 17us) server_negotiation.cc:407] Connection header received
0619 02:52:32.195394 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:52:32.195398 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:52:32.195457 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:52:32.195538 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:52:32.196872 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:32.197565 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:32.198948 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:52:32.199331 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:52:32.199421 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:52:32.203467 (+ 4046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:52:32.203481 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:52:32.203496 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:52:32.203536 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:52:32.206104 (+ 2568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:52:32.206906 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:52:32.206913 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:52:32.206916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:52:32.207070 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:52:32.207530 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:52:32.207535 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:52:32.207537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:52:32.207902 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:52:32.208112 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0619 02:52:32.208458 (+ 346us) server_negotiation.cc:299] Negotiation successful
0619 02:52:32.208619 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 02:52:32.209442 30781 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:52:32.296773 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 30658
I20260619 02:52:32.308933 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 30263
I20260619 02:52:32.310194 30325 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:52:32.419428 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 30263
I20260619 02:52:32.436856 18968 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 02:52:32.772861 18968 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 02:52:32.773101 18968 mini_ranger.cc:66] Stopping Ranger...
I20260619 02:52:33.108819 18968 mini_ranger.cc:68] Stopped Ranger
2026-06-19 02:52:33.109 UTC [29849] LOG: received smart shutdown request
2026-06-19 02:52:33.112 UTC [29849] LOG: background worker "logical replication launcher" (PID 29858) exited with exit code 1
2026-06-19 02:52:33.112 UTC [29853] LOG: shutting down
2026-06-19 02:52:33.289 UTC [29849] LOG: database system is shut down
2026-06-19T02:52:33Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49239 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:52:33 dist-test-slave-9mr0 krb5kdc[30845](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:52:33 dist-test-slave-9mr0 krb5kdc[30845](info): set up 2 sockets
Jun 19 02:52:33 dist-test-slave-9mr0 krb5kdc[30845](info): commencing operation
krb5kdc: starting...
W20260619 02:52:34.418802 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.991s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:52:34 dist-test-slave-9mr0 krb5kdc[30845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:52:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:52:34Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.134.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.134.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.134.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/rangeradmin_127.18.134.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.134.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/rangeradmin_127.18.134.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.134.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.134.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.134.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/rangerlookup_127.18.134.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.134.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/rangerlookup_127.18.134.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.134.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.134.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.134.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/HTTP_127.18.134.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.134.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/HTTP_127.18.134.20@KRBTEST.COM.keytab.
I20260619 02:52:34.721436 18968 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskUJ45gP/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/postgres -l logfile start
2026-06-19 02:52:37.206 UTC [30875] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 02:52:37.207 UTC [30875] LOG: listening on IPv4 address "127.18.134.20", port 60765
2026-06-19 02:52:37.210 UTC [30875] LOG: listening on Unix socket "/tmp/.s.PGSQL.60765"
2026-06-19 02:52:37.231 UTC [30878] LOG: database system was shut down at 2026-06-19 02:52:35 UTC
2026-06-19 02:52:37.235 UTC [30875] LOG: database system is ready to accept connections
I20260619 02:52:38.122701 18968 mini_postgres.cc:96] Postgres bound to 60765
2026-06-19 02:52:38.142 UTC [30886] FATAL: database "slave" does not exist
127.18.134.20:60765 - accepting connections
I20260619 02:52:38.143326 18968 mini_ranger.cc:161] Starting Ranger...
I20260619 02:52:38.168292 18968 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 02:52:38.698256 18968 mini_ranger.cc:87] Created ranger Postgres database
I20260619 02:52:38.698371 18968 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ranger-admin
2026-06-19 02:52:38,976 [I] DB FLAVOR :POSTGRES
2026-06-19 02:52:38,977 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 02:52:38,977 [I] Checking connection..
2026-06-19 02:52:38,977 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:60765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 02:52:39,235 [I] Checking connection passed.
2026-06-19 02:52:39,235 [I] --------- Verifying version history table ---------
2026-06-19 02:52:39,236 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:60765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:52:39,485 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 02:52:39,485 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:60765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:52:39,743 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 02:52:39,743 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 02:52:39,743 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:60765/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 02:52:39.975 UTC [30950] WARNING: there is no transaction in progress
2026-06-19 02:52:39,981 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 02:52:39,981 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:60765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 02:52:40,233 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 02:52:40,234 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 02:52:40,234 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:60765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 02:52:40,480 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:60765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 02:52:40,725 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:60765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-9mr0.c.gcp-upstream.internal','N') ;"
2026-06-19 02:52:40,969 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:60765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 02:52:41,212 [I] Table x_portal_user does not exist in database ranger
2026-06-19 02:52:41,213 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:60765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 02:52:41,469 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 02:52:41,470 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:60765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 02:52:41,725 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:60765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 02:52:41,988 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 02:52:41,988 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:60765/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 02:52:42.990 UTC [31058] WARNING: there is no transaction in progress
2026-06-19 02:52:43.006 UTC [31058] WARNING: there is no transaction in progress
2026-06-19 02:52:43.024 UTC [31058] WARNING: there is no transaction in progress
2026-06-19 02:52:43.041 UTC [31058] WARNING: there is no transaction in progress
2026-06-19 02:52:43.061 UTC [31058] WARNING: there is no transaction in progress
2026-06-19 02:52:43.085 UTC [31058] WARNING: there is no transaction in progress
2026-06-19 02:52:43.147 UTC [31058] WARNING: there is no transaction in progress
2026-06-19 02:52:43.154 UTC [31058] WARNING: there is no transaction in progress
2026-06-19 02:52:43.171 UTC [31058] WARNING: there is no transaction in progress
2026-06-19 02:52:43.182 UTC [31058] WARNING: there is no transaction in progress
2026-06-19 02:52:43.194 UTC [31058] WARNING: there is no transaction in progress
2026-06-19 02:52:43.208 UTC [31058] WARNING: there is no transaction in progress
2026-06-19 02:52:43.218 UTC [31058] WARNING: there is no transaction in progress
2026-06-19 02:52:43.225 UTC [31058] WARNING: there is no transaction in progress
2026-06-19 02:52:43.232 UTC [31058] WARNING: there is no transaction in progress
2026-06-19 02:52:43.239 UTC [31058] WARNING: there is no transaction in progress
2026-06-19 02:52:43,945 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 02:52:43,946 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:60765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 02:52:44,196 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 02:52:44,197 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:60765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 02:52:44,451 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 02:52:44,452 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:60765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 02:52:44,718 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 02:52:44,718 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:60765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 02:52:44,981 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:60765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 02:52:45,238 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:60765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-9mr0.c.gcp-upstream.internal';"
2026-06-19 02:52:45,490 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 02:52:45,491 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:60765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 02:52:45,750 [I] DB_PATCHES have already been applied
I20260619 02:52:45.756177 18968 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ranger-admin:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskUJ45gP/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 02:52:45.756268 18968 mini_ranger.cc:193] Using host: 127.18.134.20
I20260619 02:52:45.769922 18968 mini_ranger.cc:227] Ranger admin URL: http://127.18.134.20:38345
Jun 19, 2026 2:52:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 2:52:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 2:52:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 2:52:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 2:52:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 2:52:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 2:52:46 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 02:52:46.776038 18968 mini_ranger.cc:160] Time spent starting Ranger: real 8.633s user 0.005s sys 0.034s
Jun 19, 2026 2:53:07 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 2:53:07 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 19, 2026 2:53:07 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 2:53:07 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 2:53:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 2:53:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 2:53:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 2:53:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 2:53:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 2:53:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 2:53:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 2:53:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 2:53:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 2:53:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 2:53:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 2:53:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 2:53:07 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 2:53:08 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 02:53:08.807348 18968 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.134.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.134.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.134.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/rangerkms_127.18.134.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.134.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/rangerkms_127.18.134.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.134.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.134.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.134.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/HTTP_127.18.134.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.134.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/HTTP_127.18.134.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 02:53:08 dist-test-slave-9mr0 krb5kdc[30845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837588, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 02:53:08.974050 18968 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 02:53:08.996220 18968 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 02:53:09.932502 18968 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 02:53:09.932601 18968 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ranger-kms
I20260619 02:53:09.932639 18968 mini_ranger_kms.cc:227] Using postgres at 127.18.134.20:60765
2026-06-19 02:53:10,259 [I] DB FLAVOR :POSTGRES
2026-06-19 02:53:10,259 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 02:53:10,259 [I] Checking connection
2026-06-19 02:53:10,259 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:60765/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 02:53:10,509 [I] connection success
2026-06-19 02:53:10,509 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 02:53:10,509 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 02:53:10,510 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:60765/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 02:53:10,757 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 02:53:10,758 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 02:53:10,758 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 02:53:10,758 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.134.20:60765/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 02:53:11,024 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 02:53:11.529633 18968 mini_ranger_kms.cc:310] Created kms service
I20260619 02:53:11.723690 18968 mini_ranger_kms.cc:326] Created kudu user
I20260619 02:53:11.771358 18968 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 02:53:11 dist-test-slave-9mr0 krb5kdc[30845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837588, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.134.20@KRBTEST.COM
I20260619 02:53:12.168831 18968 mini_ranger_kms.cc:384] Added ranger policy
I20260619 02:53:12.169121 18968 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ranger-kms:/tmp/dist-test-taskUJ45gP/build/asan/bin/postgresql.jar:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskUJ45gP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 02:53:12.169167 18968 mini_ranger_kms.cc:240] Using host: 127.18.134.20
I20260619 02:53:12.182437 18968 mini_ranger_kms.cc:276] Ranger KMS PID: 31238
I20260619 02:53:12.182566 18968 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.134.20:36465
Jun 19, 2026 2:53:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 2:53:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 2:53:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 2:53:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 2:53:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 2:53:13 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 02:53:14.246201 18968 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.272s user 0.066s sys 0.272s
I20260619 02:53:14.246403 18968 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 02:53:14.246574 18968 mini_ranger_kms.cc:401] 127.18.134.20:36465/kms/v1/keys
Jun 19 02:53:14 dist-test-slave-9mr0 krb5kdc[30845](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781837594, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.134.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 02:53:14 dist-test-slave-9mr0 krb5kdc[30845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837594, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.134.20@KRBTEST.COM for HTTP/127.18.134.20@KRBTEST.COM
Jun 19 02:53:15 dist-test-slave-9mr0 krb5kdc[30845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837594, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.134.20@KRBTEST.COM for HTTP/127.18.134.20@KRBTEST.COM
Jun 19 02:53:17 dist-test-slave-9mr0 krb5kdc[30845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:53:17.525961 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:34193
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:36219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:34193
--ranger_config_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.134.20:36465/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:53:17.842417 31289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:17.842737 31289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:17.842878 31289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:53:17.850041 31289 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:53:17.850142 31289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:17.850193 31289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:17.850268 31289 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:53:17.850358 31289 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:53:17.860731 31289 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:36219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.134.20:36465/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:34193
--ranger_config_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:34193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:17.861989 31289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:17.863488 31289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:53:17.876335 31295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:17.876516 31294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:17.876581 31297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:17.877614 31289 server_base.cc:1034] running on GCE node
I20260619 02:53:17.878360 31289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:17.879830 31289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:17.881022 31289 hybrid_clock.cc:648] HybridClock initialized: now 1781837597880970 us; error 47 us; skew 500 ppm
I20260619 02:53:17.881338 31289 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:17 dist-test-slave-9mr0 krb5kdc[30845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837597, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:17.884307 31289 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:53:17.885769 31289 webserver.cc:460] Webserver started at http://127.18.134.62:33537/ using document root <none> and password file <none>
I20260619 02:53:17.886389 31289 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:17.886487 31289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:53:17.886811 31289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:53:17 dist-test-slave-9mr0 krb5kdc[30845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837597, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for HTTP/127.18.134.20@KRBTEST.COM
I20260619 02:53:17.942869 31289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "428a8cb6b5a844fdad3805c9772e67a0"
format_stamp: "Formatted at 2026-06-19 02:53:17 on dist-test-slave-9mr0"
server_key: "5c1e41784277ee79862e97655374c5d3"
server_key_iv: "b310b5b3459363dc12ea49dcacf8de1d"
server_key_version: "kuduclusterkey@0"
I20260619 02:53:17.943965 31289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "428a8cb6b5a844fdad3805c9772e67a0"
format_stamp: "Formatted at 2026-06-19 02:53:17 on dist-test-slave-9mr0"
server_key: "5c1e41784277ee79862e97655374c5d3"
server_key_iv: "b310b5b3459363dc12ea49dcacf8de1d"
server_key_version: "kuduclusterkey@0"
I20260619 02:53:17.950789 31289 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260619 02:53:17.968608 31304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:17.970117 31289 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260619 02:53:17.970306 31289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "428a8cb6b5a844fdad3805c9772e67a0"
format_stamp: "Formatted at 2026-06-19 02:53:17 on dist-test-slave-9mr0"
server_key: "5c1e41784277ee79862e97655374c5d3"
server_key_iv: "b310b5b3459363dc12ea49dcacf8de1d"
server_key_version: "kuduclusterkey@0"
I20260619 02:53:17.970530 31289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:17.986126 31289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:17.990219 31289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:17.990502 31289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:18.008430 31289 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:34193
I20260619 02:53:18.008435 31355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:34193 every 8 connection(s)
I20260619 02:53:18.009740 31289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:53:18.013461 31356 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:53:18.017414 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 31289
I20260619 02:53:18.017607 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:53:18.018389 18968 external_mini_cluster.cc:1428] Setting key 76346b52685dc453ac04bd4f795eeff9
I20260619 02:53:18.026317 31356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0: Bootstrap starting.
I20260619 02:53:18.030589 31356 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0: Neither blocks nor log segments found. Creating new log.
I20260619 02:53:18.032089 31356 log.cc:824] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0: Log is configured to *not* fsync() on all Append() calls
I20260619 02:53:18.039422 31356 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0: No bootstrap required, opened a new log
I20260619 02:53:18.048218 31356 raft_consensus.cc:348] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428a8cb6b5a844fdad3805c9772e67a0" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 34193 } }
I20260619 02:53:18.048622 31356 raft_consensus.cc:374] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:53:18.048725 31356 raft_consensus.cc:729] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 428a8cb6b5a844fdad3805c9772e67a0, State: Initialized, Role: FOLLOWER
Jun 19 02:53:18 dist-test-slave-9mr0 krb5kdc[30845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:53:18.049417 31356 consensus_queue.cc:260] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428a8cb6b5a844fdad3805c9772e67a0" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 34193 } }
I20260619 02:53:18.049690 31356 raft_consensus.cc:388] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:53:18.049836 31356 raft_consensus.cc:482] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:53:18.050014 31356 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:53:18.052532 31356 raft_consensus.cc:504] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428a8cb6b5a844fdad3805c9772e67a0" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 34193 } }
I20260619 02:53:18.053112 31356 leader_election.cc:302] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 428a8cb6b5a844fdad3805c9772e67a0; no voters:
I20260619 02:53:18.053718 31356 leader_election.cc:288] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:53:18.053838 31361 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:53:18.057866 31361 raft_consensus.cc:686] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [term 1 LEADER]: Becoming Leader. State: Replica: 428a8cb6b5a844fdad3805c9772e67a0, State: Running, Role: LEADER
I20260619 02:53:18.058568 31361 consensus_queue.cc:237] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428a8cb6b5a844fdad3805c9772e67a0" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 34193 } }
I20260619 02:53:18.060541 31359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:18.020463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53252 (local address 127.18.134.62:34193)
0619 02:53:18.035224 (+ 14761us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:18.035243 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:18.035299 (+ 56us) server_negotiation.cc:407] Connection header received
0619 02:53:18.037638 (+ 2339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:18.037668 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:18.038208 (+ 540us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:18.038693 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:18.040219 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:18.042249 (+ 2030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:18.044093 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:18.045138 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:18.045315 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:18.050430 (+ 5115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:18.050442 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:18.050464 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:18.050521 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:18.054756 (+ 4235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:18.055814 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:18.055823 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:18.055832 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:18.056024 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:18.056442 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:18.056446 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:18.056450 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:18.057161 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:18.058488 (+ 1327us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:18.058791 (+ 303us) server_negotiation.cc:299] Negotiation successful
0619 02:53:18.059198 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14491,"thread_start_us":15627,"threads_started":1}
I20260619 02:53:18.064913 31356 sys_catalog.cc:564] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:53:18.066879 31362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "428a8cb6b5a844fdad3805c9772e67a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428a8cb6b5a844fdad3805c9772e67a0" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 34193 } } }
I20260619 02:53:18.067237 31363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 428a8cb6b5a844fdad3805c9772e67a0. Latest consensus state: current_term: 1 leader_uuid: "428a8cb6b5a844fdad3805c9772e67a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428a8cb6b5a844fdad3805c9772e67a0" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 34193 } } }
I20260619 02:53:18.067610 31363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [sys.catalog]: This master's current role is: LEADER
I20260619 02:53:18.067625 31362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [sys.catalog]: This master's current role is: LEADER
I20260619 02:53:18.069974 31368 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:53:18.076757 31368 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:53:18.097056 31368 catalog_manager.cc:1347] Generated new cluster ID: a61ae0da87104caa984ebb743c96185c
I20260619 02:53:18.097196 31368 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:53:18.123558 31356 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 02:53:18.125384 31368 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:53:18.127369 31368 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:53:18.145278 31368 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0: Generated new TSK 0
I20260619 02:53:18.146974 31368 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.134.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
I20260619 02:53:18.970729 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.134.20:36465/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:0
--local_ip_for_outbound_sockets=127.18.134.1
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:34193
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:36219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:53:19 dist-test-slave-9mr0 krb5kdc[30845](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781837599, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260619 02:53:19.372768 31402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:19.373181 31402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:19.373358 31402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:53:19.384801 31402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:19.384956 31402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:19.385207 31402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:53:19.397828 31402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:36219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.134.20:36465/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:34193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:19.399219 31402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:19.400462 31402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:53:19.417428 31412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:19.417434 31413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:19.419706 31415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:19.420720 31402 server_base.cc:1034] running on GCE node
I20260619 02:53:19.421494 31402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:19.422736 31402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:19.424010 31402 hybrid_clock.cc:648] HybridClock initialized: now 1781837599423945 us; error 55 us; skew 500 ppm
I20260619 02:53:19.424458 31402 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:19 dist-test-slave-9mr0 krb5kdc[30845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837599, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:19.428727 31402 init.cc:377] Logged in from keytab as kudu/127.18.134.1@KRBTEST.COM (short username kudu)
I20260619 02:53:19.431286 31402 webserver.cc:460] Webserver started at http://127.18.134.1:34675/ using document root <none> and password file <none>
I20260619 02:53:19.432266 31402 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:19.432402 31402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:53:19.432726 31402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:53:19 dist-test-slave-9mr0 krb5kdc[30845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837599, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for HTTP/127.18.134.20@KRBTEST.COM
I20260619 02:53:19.453223 31402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/data/instance:
uuid: "4f182f97d3c24d2e8995592610ae5943"
format_stamp: "Formatted at 2026-06-19 02:53:19 on dist-test-slave-9mr0"
server_key: "dffe14b63af31e681e2d10a0dfce48fd"
server_key_iv: "115f77b28cd40dd63b62e8992981a9be"
server_key_version: "kuduclusterkey@0"
I20260619 02:53:19.454330 31402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance:
uuid: "4f182f97d3c24d2e8995592610ae5943"
format_stamp: "Formatted at 2026-06-19 02:53:19 on dist-test-slave-9mr0"
server_key: "dffe14b63af31e681e2d10a0dfce48fd"
server_key_iv: "115f77b28cd40dd63b62e8992981a9be"
server_key_version: "kuduclusterkey@0"
I20260619 02:53:19.476806 31402 fs_manager.cc:691] Time spent creating directory manager: real 0.022s user 0.000s sys 0.020s
I20260619 02:53:19.507277 31422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:19.512208 31402 fs_manager.cc:725] Time spent opening block manager: real 0.019s user 0.012s sys 0.004s
I20260619 02:53:19.512533 31402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "4f182f97d3c24d2e8995592610ae5943"
format_stamp: "Formatted at 2026-06-19 02:53:19 on dist-test-slave-9mr0"
server_key: "dffe14b63af31e681e2d10a0dfce48fd"
server_key_iv: "115f77b28cd40dd63b62e8992981a9be"
server_key_version: "kuduclusterkey@0"
I20260619 02:53:19.512918 31402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:19.561002 31402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:19.576095 31402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:19.576535 31402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:19.590413 31402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:53:19.592549 31402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:53:19.592660 31402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:19.592794 31402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:53:19.592859 31402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 02:53:19 dist-test-slave-9mr0 krb5kdc[30845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837599, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for HTTP/127.18.134.20@KRBTEST.COM
Jun 19 02:53:20 dist-test-slave-9mr0 krb5kdc[30845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837599, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for HTTP/127.18.134.20@KRBTEST.COM
I20260619 02:53:20.288111 31402 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:42525
I20260619 02:53:20.288141 31535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:42525 every 8 connection(s)
I20260619 02:53:20.289805 31402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:53:20.294616 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 31402
I20260619 02:53:20.294844 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance
I20260619 02:53:20.295555 18968 external_mini_cluster.cc:1428] Setting key f5d43e9c10d9344234073a8af5e462d7
WARNING: no policy specified for kudu/127.18.134.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.2@KRBTEST.COM" created.
Jun 19 02:53:20 dist-test-slave-9mr0 krb5kdc[30845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837599, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:53:20.381732 31540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:20.335594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:34193 (local address 127.18.134.1:49183)
0619 02:53:20.339814 (+ 4220us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:20.339863 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:20.347305 (+ 7442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:20.348461 (+ 1156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:20.348474 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:20.349084 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:53:20.349911 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:20.349937 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:20.364343 (+ 14406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:20.364349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:20.366695 (+ 2346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:20.366721 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:20.368028 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:20.368035 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:20.368326 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:20.369194 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:53:20.369229 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:53:20.372172 (+ 2943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:53:20.376642 (+ 4470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:20.376654 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:20.376675 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:20.377321 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:20.377804 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:20.377809 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:20.377813 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:20.378114 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:20.378923 (+ 809us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:53:20.378932 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:53:20.379407 (+ 475us) client_negotiation.cc:769] Sending connection context
0619 02:53:20.379730 (+ 323us) client_negotiation.cc:240] Negotiation successful
0619 02:53:20.380308 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3915,"thread_start_us":4219,"threads_started":1}
I20260619 02:53:20.382959 31541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:20.336265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:49183 (local address 127.18.134.62:34193)
0619 02:53:20.345700 (+ 9435us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:20.345707 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:20.345732 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:53:20.347646 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:20.347653 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:20.347794 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:20.347970 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:20.362836 (+ 14866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:20.364052 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:20.366886 (+ 2834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:20.367547 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:20.367650 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:20.372457 (+ 4807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:20.372463 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:20.372467 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:20.372520 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:20.376442 (+ 3922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:20.377487 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:20.377493 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:20.377497 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:20.377663 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:20.378262 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:20.378267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:20.378271 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:20.378672 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:20.378825 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:20.382286 (+ 3461us) server_negotiation.cc:299] Negotiation successful
0619 02:53:20.382546 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9308,"thread_start_us":9190,"threads_started":1}
I20260619 02:53:20.385322 31537 heartbeater.cc:347] Connected to a master server at 127.18.134.62:34193
I20260619 02:53:20.385651 31537 heartbeater.cc:464] Registering TS with master...
I20260619 02:53:20.386528 31537 heartbeater.cc:511] Master 127.18.134.62:34193 requested a full tablet report, sending...
I20260619 02:53:20.389200 31321 ts_manager.cc:194] Registered new tserver with Master: 4f182f97d3c24d2e8995592610ae5943 (127.18.134.1:42525)
I20260619 02:53:20.391260 31321 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.1@KRBTEST.COM'} at 127.18.134.1:49183
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
W20260619 02:53:20.404618 18968 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.18.134.2: real 0.106s user 0.007s sys 0.011s
I20260619 02:53:20.405504 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.134.20:36465/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:0
--local_ip_for_outbound_sockets=127.18.134.2
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:34193
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:36219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:53:20.761435 31543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:20.761735 31543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:20.761844 31543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:53:20.769125 31543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:20.769249 31543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:20.769398 31543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:53:20.779563 31543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:36219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.134.20:36465/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:34193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:20.780830 31543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:20.782049 31543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:53:20.795068 31556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:20.795207 31555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:20.795744 31558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:20.796159 31543 server_base.cc:1034] running on GCE node
I20260619 02:53:20.796756 31543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:20.797679 31543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:20.798889 31543 hybrid_clock.cc:648] HybridClock initialized: now 1781837600798780 us; error 98 us; skew 500 ppm
I20260619 02:53:20.799204 31543 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:20 dist-test-slave-9mr0 krb5kdc[30845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837600, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:20.802146 31543 init.cc:377] Logged in from keytab as kudu/127.18.134.2@KRBTEST.COM (short username kudu)
I20260619 02:53:20.803596 31543 webserver.cc:460] Webserver started at http://127.18.134.2:38321/ using document root <none> and password file <none>
I20260619 02:53:20.804262 31543 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:20.804375 31543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:53:20.804643 31543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:53:20 dist-test-slave-9mr0 krb5kdc[30845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837600, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for HTTP/127.18.134.20@KRBTEST.COM
I20260619 02:53:20.820744 31543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/data/instance:
uuid: "1cd4a86f7f0c4de8a71d50e53f14e249"
format_stamp: "Formatted at 2026-06-19 02:53:20 on dist-test-slave-9mr0"
server_key: "ec690fc916b8680095dac342e19f84b8"
server_key_iv: "8404e587d51bff49e9f404c9bfc9230a"
server_key_version: "kuduclusterkey@0"
I20260619 02:53:20.821962 31543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance:
uuid: "1cd4a86f7f0c4de8a71d50e53f14e249"
format_stamp: "Formatted at 2026-06-19 02:53:20 on dist-test-slave-9mr0"
server_key: "ec690fc916b8680095dac342e19f84b8"
server_key_iv: "8404e587d51bff49e9f404c9bfc9230a"
server_key_version: "kuduclusterkey@0"
I20260619 02:53:20.829028 31543 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.004s
I20260619 02:53:20.845551 31565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:20.847150 31543 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:53:20.847357 31543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "1cd4a86f7f0c4de8a71d50e53f14e249"
format_stamp: "Formatted at 2026-06-19 02:53:20 on dist-test-slave-9mr0"
server_key: "ec690fc916b8680095dac342e19f84b8"
server_key_iv: "8404e587d51bff49e9f404c9bfc9230a"
server_key_version: "kuduclusterkey@0"
I20260619 02:53:20.847584 31543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:20.866030 31543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:20.869575 31543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:20.869850 31543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:20.870877 31543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:53:20.872212 31543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:53:20.872296 31543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:20.872416 31543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:53:20.872468 31543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:20.901647 31543 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:45787
I20260619 02:53:20.901667 31677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:45787 every 8 connection(s)
I20260619 02:53:20.902999 31543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:53:20.912997 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 31543
I20260619 02:53:20.913151 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance
I20260619 02:53:20.913574 18968 external_mini_cluster.cc:1428] Setting key c64325e33c92422abff0e968cbb5ae92
Jun 19 02:53:20 dist-test-slave-9mr0 krb5kdc[30845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837600, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:53:20.921681 31680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:20.905444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:47441 (local address 127.18.134.62:34193)
0619 02:53:20.905903 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:20.905909 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:20.906877 (+ 968us) server_negotiation.cc:407] Connection header received
0619 02:53:20.908682 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:20.908687 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:20.908755 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:20.908852 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:20.910254 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:20.911060 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:20.912531 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:20.912963 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:20.913080 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:20.916182 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:20.916187 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:20.916191 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:20.916222 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:20.918757 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:20.919459 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:20.919466 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:20.919470 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:20.919597 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:20.920020 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:20.920026 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:20.920029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:20.920301 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:20.920452 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:20.921097 (+ 645us) server_negotiation.cc:299] Negotiation successful
0619 02:53:20.921321 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":203,"threads_started":1}
I20260619 02:53:20.922572 31681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:20.905977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:34193 (local address 127.18.134.2:47441)
0619 02:53:20.906711 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:20.906758 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:20.908415 (+ 1657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:20.909090 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:20.909098 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:20.909535 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:53:20.910104 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:20.910120 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:20.911175 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:20.911178 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:20.912397 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:20.912409 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:20.913195 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:20.913203 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:20.913420 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:20.914028 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:53:20.914054 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:53:20.915983 (+ 1929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:53:20.918897 (+ 2914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:20.918906 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:20.918924 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:20.919336 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:20.919705 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:20.919710 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:20.919714 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:20.919916 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:20.920418 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:53:20.920426 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:53:20.920752 (+ 326us) client_negotiation.cc:769] Sending connection context
0619 02:53:20.921001 (+ 249us) client_negotiation.cc:240] Negotiation successful
0619 02:53:20.921487 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":272,"threads_started":1}
I20260619 02:53:20.924041 31678 heartbeater.cc:347] Connected to a master server at 127.18.134.62:34193
I20260619 02:53:20.924309 31678 heartbeater.cc:464] Registering TS with master...
I20260619 02:53:20.924969 31678 heartbeater.cc:511] Master 127.18.134.62:34193 requested a full tablet report, sending...
I20260619 02:53:20.926453 31321 ts_manager.cc:194] Registered new tserver with Master: 1cd4a86f7f0c4de8a71d50e53f14e249 (127.18.134.2:45787)
I20260619 02:53:20.927434 31321 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.2@KRBTEST.COM'} at 127.18.134.2:47441
WARNING: no policy specified for kudu/127.18.134.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
I20260619 02:53:20.960068 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.134.20:36465/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:0
--local_ip_for_outbound_sockets=127.18.134.3
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:34193
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:36219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:53:21.280333 31684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:21.280596 31684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:21.280684 31684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:53:21.287925 31684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:21.288035 31684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:21.288156 31684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:53:21.297891 31684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:36219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.134.20:36465/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:34193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:21.299235 31684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:21.300460 31684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:53:21.312680 31690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:21.312752 31689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:21.312703 31692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:21.313230 31684 server_base.cc:1034] running on GCE node
I20260619 02:53:21.313874 31684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:21.314855 31684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:21.316066 31684 hybrid_clock.cc:648] HybridClock initialized: now 1781837601316011 us; error 49 us; skew 500 ppm
I20260619 02:53:21.316373 31684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:21 dist-test-slave-9mr0 krb5kdc[30845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837601, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:21.319561 31684 init.cc:377] Logged in from keytab as kudu/127.18.134.3@KRBTEST.COM (short username kudu)
I20260619 02:53:21.320966 31684 webserver.cc:460] Webserver started at http://127.18.134.3:40823/ using document root <none> and password file <none>
I20260619 02:53:21.321658 31684 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:21.321758 31684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:53:21.322028 31684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 02:53:21 dist-test-slave-9mr0 krb5kdc[30845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837601, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for HTTP/127.18.134.20@KRBTEST.COM
I20260619 02:53:21.335788 31684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/data/instance:
uuid: "744815233d3245b582b01585c7f478a9"
format_stamp: "Formatted at 2026-06-19 02:53:21 on dist-test-slave-9mr0"
server_key: "d6a12027c84b6fb01c9656f0dc7034a0"
server_key_iv: "da330b1e7d18c1f07bab54be6d42fbe0"
server_key_version: "kuduclusterkey@0"
I20260619 02:53:21.336621 31684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance:
uuid: "744815233d3245b582b01585c7f478a9"
format_stamp: "Formatted at 2026-06-19 02:53:21 on dist-test-slave-9mr0"
server_key: "d6a12027c84b6fb01c9656f0dc7034a0"
server_key_iv: "da330b1e7d18c1f07bab54be6d42fbe0"
server_key_version: "kuduclusterkey@0"
I20260619 02:53:21.342161 31684 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260619 02:53:21.356134 31699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:21.357621 31684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:53:21.357774 31684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "744815233d3245b582b01585c7f478a9"
format_stamp: "Formatted at 2026-06-19 02:53:21 on dist-test-slave-9mr0"
server_key: "d6a12027c84b6fb01c9656f0dc7034a0"
server_key_iv: "da330b1e7d18c1f07bab54be6d42fbe0"
server_key_version: "kuduclusterkey@0"
I20260619 02:53:21.357990 31684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:21.375560 31684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:21.379153 31684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:21.379420 31684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:21.380363 31684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:53:21.381706 31684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:53:21.381778 31684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:21.381886 31684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:53:21.381935 31684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:21.395431 31537 heartbeater.cc:503] Master 127.18.134.62:34193 was elected leader, sending a full tablet report...
I20260619 02:53:21.411458 31684 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:42699
I20260619 02:53:21.411475 31811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:42699 every 8 connection(s)
I20260619 02:53:21.412678 31684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:53:21.421589 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 31684
I20260619 02:53:21.421756 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance
I20260619 02:53:21.422224 18968 external_mini_cluster.cc:1428] Setting key fc8b0a0de261459a36bc7cdaf65a1e8a
Jun 19 02:53:21 dist-test-slave-9mr0 krb5kdc[30845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837601, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:53:21.431133 31680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:21.415167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:53665 (local address 127.18.134.62:34193)
0619 02:53:21.415351 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:21.415356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:21.416403 (+ 1047us) server_negotiation.cc:407] Connection header received
0619 02:53:21.418207 (+ 1804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:21.418213 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:21.418323 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:21.418405 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:21.419790 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:21.420613 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:21.422190 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:21.422657 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:21.422792 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:21.426106 (+ 3314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:21.426111 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:21.426114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:21.426150 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:21.428576 (+ 2426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:21.429234 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:21.429237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:21.429239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:21.429356 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:21.429709 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:21.429712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:21.429714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:21.429972 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:21.430079 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:21.430739 (+ 660us) server_negotiation.cc:299] Negotiation successful
0619 02:53:21.430919 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 02:53:21.432026 31814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:21.415595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:34193 (local address 127.18.134.3:53665)
0619 02:53:21.416251 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:21.416291 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:21.417942 (+ 1651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:21.418668 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:21.418679 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:21.419099 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:53:21.419637 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:21.419654 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:21.420755 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:21.420759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:21.422065 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:21.422076 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:21.422867 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:21.422874 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:21.423140 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:21.423798 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:53:21.423819 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:53:21.425906 (+ 2087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:53:21.428699 (+ 2793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:21.428706 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:21.428725 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:21.429142 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:21.429438 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:21.429441 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:21.429444 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:21.429631 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:21.430094 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:53:21.430100 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:53:21.430400 (+ 300us) client_negotiation.cc:769] Sending connection context
0619 02:53:21.430621 (+ 221us) client_negotiation.cc:240] Negotiation successful
0619 02:53:21.431031 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":261,"threads_started":1}
I20260619 02:53:21.433279 31812 heartbeater.cc:347] Connected to a master server at 127.18.134.62:34193
I20260619 02:53:21.433535 31812 heartbeater.cc:464] Registering TS with master...
I20260619 02:53:21.434154 31812 heartbeater.cc:511] Master 127.18.134.62:34193 requested a full tablet report, sending...
I20260619 02:53:21.435475 31321 ts_manager.cc:194] Registered new tserver with Master: 744815233d3245b582b01585c7f478a9 (127.18.134.3:42699)
I20260619 02:53:21.436319 31321 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.3@KRBTEST.COM'} at 127.18.134.3:53665
I20260619 02:53:21.437544 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:53:21.439180 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 31402
Jun 19 02:53:21 dist-test-slave-9mr0 krb5kdc[30845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.1@KRBTEST.COM
I20260619 02:53:21.454728 31816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:21.440015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38264 (local address 127.18.134.1:42525)
0619 02:53:21.440553 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:21.440559 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:21.440592 (+ 33us) server_negotiation.cc:407] Connection header received
0619 02:53:21.440645 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:21.440650 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:21.440832 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:21.440968 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:21.442476 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:21.443607 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:21.444996 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:21.445515 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:21.445604 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:21.449545 (+ 3941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:21.449555 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:21.449569 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:21.449603 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:21.452284 (+ 2681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:21.452960 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:21.452966 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:21.452969 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:21.453103 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:21.453469 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:21.453472 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:21.453474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:21.453782 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:21.454000 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:21.454223 (+ 223us) server_negotiation.cc:299] Negotiation successful
0619 02:53:21.454415 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":262,"threads_started":1}
I20260619 02:53:21.455966 31531 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:53:21.537724 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 31402
I20260619 02:53:21.549657 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 31543
Jun 19 02:53:21 dist-test-slave-9mr0 krb5kdc[30845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.2@KRBTEST.COM
I20260619 02:53:21.565205 31818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:21.550495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40668 (local address 127.18.134.2:45787)
0619 02:53:21.551029 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:21.551036 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:21.551061 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:53:21.551117 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:21.551123 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:21.551326 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:21.551453 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:21.552657 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:21.553600 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:21.555011 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:21.555572 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:21.555680 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:21.559966 (+ 4286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:21.559978 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:21.559994 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:21.560029 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:21.562681 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:21.563395 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:21.563401 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:21.563403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:21.563530 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:21.563837 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:21.563840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:21.563842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:21.564167 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:21.564382 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:21.564674 (+ 292us) server_negotiation.cc:299] Negotiation successful
0619 02:53:21.564865 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":297,"threads_started":1}
I20260619 02:53:21.566488 31673 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:53:21.646607 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 31543
I20260619 02:53:21.658337 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 31684
Jun 19 02:53:21 dist-test-slave-9mr0 krb5kdc[30845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.3@KRBTEST.COM
I20260619 02:53:21.674365 31820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:21.659243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35640 (local address 127.18.134.3:42699)
0619 02:53:21.659837 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:21.659844 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:21.659872 (+ 28us) server_negotiation.cc:407] Connection header received
0619 02:53:21.659927 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:21.659933 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:21.660137 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:21.660264 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:21.661473 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:21.662541 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:21.663853 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:21.664446 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:21.664594 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:21.668765 (+ 4171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:21.668776 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:21.668790 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:21.668824 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:21.671832 (+ 3008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:21.672440 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:21.672447 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:21.672449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:21.672596 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:21.672945 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:21.672948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:21.672951 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:21.673289 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:21.673518 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:21.673731 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 02:53:21.673948 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":317,"threads_started":1}
I20260619 02:53:21.675633 31807 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:53:21.756455 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 31684
I20260619 02:53:21.768502 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 31289
I20260619 02:53:21.769650 31351 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:53:21.872503 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 31289
I20260619 02:53:21.894752 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:34193
--webserver_interface=127.18.134.62
--webserver_port=33537
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:36219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:34193
--ranger_config_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.134.20:36465/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.134.20:36465/kms with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:53:22.205870 31823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:22.206157 31823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:22.206279 31823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:53:22.213505 31823 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:53:22.213604 31823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:22.213675 31823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:22.213747 31823 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:53:22.213812 31823 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:53:22.223773 31823 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:36219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.134.20:36465/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:34193
--ranger_config_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:34193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=33537
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:22.225021 31823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:22.226320 31823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:53:22.237776 31828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:22.237709 31831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:22.237741 31829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:22.238734 31823 server_base.cc:1034] running on GCE node
I20260619 02:53:22.239547 31823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:22.240973 31823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:22.242213 31823 hybrid_clock.cc:648] HybridClock initialized: now 1781837602242154 us; error 47 us; skew 500 ppm
I20260619 02:53:22.242564 31823 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:22 dist-test-slave-9mr0 krb5kdc[30845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837602, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:22.245493 31823 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:53:22.246935 31823 webserver.cc:460] Webserver started at http://127.18.134.62:33537/ using document root <none> and password file <none>
I20260619 02:53:22.247542 31823 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:22.247661 31823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 02:53:22 dist-test-slave-9mr0 krb5kdc[30845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837602, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for HTTP/127.18.134.20@KRBTEST.COM
I20260619 02:53:22.298976 31823 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.004s
I20260619 02:53:22.302139 31839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:22.303635 31823 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:53:22.303799 31823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "428a8cb6b5a844fdad3805c9772e67a0"
format_stamp: "Formatted at 2026-06-19 02:53:17 on dist-test-slave-9mr0"
server_key: "5c1e41784277ee79862e97655374c5d3"
server_key_iv: "b310b5b3459363dc12ea49dcacf8de1d"
server_key_version: "kuduclusterkey@0"
I20260619 02:53:22.304744 31823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:22.321062 31823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:22.324672 31823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:22.324929 31823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:22.341982 31823 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:34193
I20260619 02:53:22.341989 31890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:34193 every 8 connection(s)
I20260619 02:53:22.343449 31823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:53:22.345779 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 31823
I20260619 02:53:22.346536 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.134.20:36465/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:42525
--local_ip_for_outbound_sockets=127.18.134.1
--tserver_master_addrs=127.18.134.62:34193
--webserver_port=34675
--webserver_interface=127.18.134.1
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:36219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.134.20:36465/kms with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:53:22.350121 31891 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:53:22.353328 31891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0: Bootstrap starting.
I20260619 02:53:22.371888 31891 log.cc:824] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0: Log is configured to *not* fsync() on all Append() calls
I20260619 02:53:22.380990 31891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:53:22.381522 31891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0: Bootstrap complete.
I20260619 02:53:22.390503 31891 raft_consensus.cc:348] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428a8cb6b5a844fdad3805c9772e67a0" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 34193 } }
I20260619 02:53:22.391444 31891 raft_consensus.cc:729] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 428a8cb6b5a844fdad3805c9772e67a0, State: Initialized, Role: FOLLOWER
I20260619 02:53:22.392045 31891 consensus_queue.cc:260] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428a8cb6b5a844fdad3805c9772e67a0" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 34193 } }
I20260619 02:53:22.392336 31891 raft_consensus.cc:388] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:53:22.392449 31891 raft_consensus.cc:482] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:53:22.392618 31891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:53:22.396534 31891 raft_consensus.cc:504] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428a8cb6b5a844fdad3805c9772e67a0" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 34193 } }
I20260619 02:53:22.397014 31891 leader_election.cc:302] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 428a8cb6b5a844fdad3805c9772e67a0; no voters:
I20260619 02:53:22.397635 31891 leader_election.cc:288] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 02:53:22.397792 31895 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 02:53:22.398916 31895 raft_consensus.cc:686] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [term 2 LEADER]: Becoming Leader. State: Replica: 428a8cb6b5a844fdad3805c9772e67a0, State: Running, Role: LEADER
I20260619 02:53:22.399565 31895 consensus_queue.cc:237] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428a8cb6b5a844fdad3805c9772e67a0" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 34193 } }
I20260619 02:53:22.399961 31891 sys_catalog.cc:564] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:53:22.402091 31896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "428a8cb6b5a844fdad3805c9772e67a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428a8cb6b5a844fdad3805c9772e67a0" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 34193 } } }
I20260619 02:53:22.402501 31896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [sys.catalog]: This master's current role is: LEADER
I20260619 02:53:22.406350 31897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 428a8cb6b5a844fdad3805c9772e67a0. Latest consensus state: current_term: 2 leader_uuid: "428a8cb6b5a844fdad3805c9772e67a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "428a8cb6b5a844fdad3805c9772e67a0" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 34193 } } }
I20260619 02:53:22.406834 31897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0 [sys.catalog]: This master's current role is: LEADER
I20260619 02:53:22.408598 31901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:53:22.414909 31901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:53:22.416311 31901 catalog_manager.cc:1259] Loaded cluster ID: a61ae0da87104caa984ebb743c96185c
I20260619 02:53:22.416486 31901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:53:22.420536 31901 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:53:22.421682 31891 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 02:53:22.422447 31901 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0: Loaded TSK: 0
I20260619 02:53:22.424350 31901 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 02:53:22.731491 31893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:22.731961 31893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:22.732168 31893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:53:22.741722 31893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:22.741842 31893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:22.742028 31893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:53:22.753893 31893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:36219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.134.20:36465/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:42525
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=34675
--tserver_master_addrs=127.18.134.62:34193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:22.755730 31893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:22.757308 31893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:53:22.782915 31928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:22.787205 31929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:22.789985 31931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:22.790850 31893 server_base.cc:1034] running on GCE node
I20260619 02:53:22.791786 31893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:22.796697 31893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:22.798009 31893 hybrid_clock.cc:648] HybridClock initialized: now 1781837602797875 us; error 132 us; skew 500 ppm
I20260619 02:53:22.798478 31893 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:22 dist-test-slave-9mr0 krb5kdc[30845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837602, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:22.802934 31893 init.cc:377] Logged in from keytab as kudu/127.18.134.1@KRBTEST.COM (short username kudu)
I20260619 02:53:22.815039 31893 webserver.cc:460] Webserver started at http://127.18.134.1:34675/ using document root <none> and password file <none>
I20260619 02:53:22.815831 31893 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:22.815986 31893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 02:53:22 dist-test-slave-9mr0 krb5kdc[30845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837602, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for HTTP/127.18.134.20@KRBTEST.COM
I20260619 02:53:22.864629 31893 fs_manager.cc:709] Time spent opening directory manager: real 0.018s user 0.012s sys 0.004s
I20260619 02:53:22.873660 31939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:22.875437 31893 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.006s sys 0.000s
I20260619 02:53:22.875618 31893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "4f182f97d3c24d2e8995592610ae5943"
format_stamp: "Formatted at 2026-06-19 02:53:19 on dist-test-slave-9mr0"
server_key: "dffe14b63af31e681e2d10a0dfce48fd"
server_key_iv: "115f77b28cd40dd63b62e8992981a9be"
server_key_version: "kuduclusterkey@0"
I20260619 02:53:22.876657 31893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:22.893242 31893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:22.897208 31893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:22.897482 31893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:22.898517 31893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:53:22.900152 31893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:53:22.900245 31893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:22.900390 31893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:53:22.900470 31893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260619 02:53:23.011762 31893 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:42525
I20260619 02:53:23.011811 32051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:42525 every 8 connection(s)
I20260619 02:53:23.013236 31893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:53:23.021565 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 31893
I20260619 02:53:23.022394 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.134.20:36465/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:45787
--local_ip_for_outbound_sockets=127.18.134.2
--tserver_master_addrs=127.18.134.62:34193
--webserver_port=38321
--webserver_interface=127.18.134.2
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:36219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.134.20:36465/kms with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:53:23 dist-test-slave-9mr0 krb5kdc[30845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837602, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:53:23.065932 32055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:23.017785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:34193 (local address 127.18.134.1:52065)
0619 02:53:23.036533 (+ 18748us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:23.036598 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:23.039334 (+ 2736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:23.041533 (+ 2199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:23.041545 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:23.042167 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:53:23.042944 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:23.042977 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:23.045472 (+ 2495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:23.045479 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:23.047481 (+ 2002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:23.047501 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:23.049467 (+ 1966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:23.049476 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:23.049765 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:23.052030 (+ 2265us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:53:23.052066 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:53:23.054924 (+ 2858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:53:23.060552 (+ 5628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:23.060563 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:23.060580 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:23.061261 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:23.061822 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:23.061827 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:23.061830 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:23.062118 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:23.063340 (+ 1222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:53:23.063348 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:53:23.063784 (+ 436us) client_negotiation.cc:769] Sending connection context
0619 02:53:23.064109 (+ 325us) client_negotiation.cc:240] Negotiation successful
0619 02:53:23.064713 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18395,"thread_start_us":18107,"threads_started":1}
I20260619 02:53:23.070554 32056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:23.019024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:52065 (local address 127.18.134.62:34193)
0619 02:53:23.035794 (+ 16770us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:23.035820 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:23.036784 (+ 964us) server_negotiation.cc:407] Connection header received
0619 02:53:23.040289 (+ 3505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:23.040311 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:23.040881 (+ 570us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:23.041346 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:23.043218 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:23.045266 (+ 2048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:23.047711 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:23.048823 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:23.049030 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:23.055337 (+ 6307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:23.055352 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:23.055376 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:23.055435 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:23.060312 (+ 4877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:23.061450 (+ 1138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:23.061459 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:23.061467 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:23.061681 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:23.062265 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:23.062270 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:23.062274 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:23.063053 (+ 779us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:23.063262 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:23.068722 (+ 5460us) server_negotiation.cc:299] Negotiation successful
0619 02:53:23.069219 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16444,"thread_start_us":16176,"threads_started":1}
I20260619 02:53:23.073282 32052 heartbeater.cc:347] Connected to a master server at 127.18.134.62:34193
I20260619 02:53:23.073596 32052 heartbeater.cc:464] Registering TS with master...
I20260619 02:53:23.074409 32052 heartbeater.cc:511] Master 127.18.134.62:34193 requested a full tablet report, sending...
W20260619 02:53:23.077255 32052 heartbeater.cc:644] Failed to heartbeat to 127.18.134.62:34193 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 02:53:23.077473 32052 heartbeater.cc:464] Registering TS with master...
I20260619 02:53:23.077744 32052 heartbeater.cc:511] Master 127.18.134.62:34193 requested a full tablet report, sending...
I20260619 02:53:23.078789 32052 heartbeater.cc:464] Registering TS with master...
I20260619 02:53:23.079037 32052 heartbeater.cc:511] Master 127.18.134.62:34193 requested a full tablet report, sending...
W20260619 02:53:23.079989 32052 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:53:23.480564 32057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:23.480985 32057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:23.481159 32057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:53:23.492653 32057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:23.492797 32057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:23.492986 32057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:53:23.508630 32057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:36219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.134.20:36465/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:45787
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=38321
--tserver_master_addrs=127.18.134.62:34193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:23.510342 32057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:23.512120 32057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:53:23.528589 32070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:23.529037 32072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:23.529342 32069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:23.529500 32057 server_base.cc:1034] running on GCE node
I20260619 02:53:23.530167 32057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:23.531251 32057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:23.532480 32057 hybrid_clock.cc:648] HybridClock initialized: now 1781837603532411 us; error 61 us; skew 500 ppm
I20260619 02:53:23.532806 32057 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:23 dist-test-slave-9mr0 krb5kdc[30845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837603, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:23.536000 32057 init.cc:377] Logged in from keytab as kudu/127.18.134.2@KRBTEST.COM (short username kudu)
I20260619 02:53:23.537720 32057 webserver.cc:460] Webserver started at http://127.18.134.2:38321/ using document root <none> and password file <none>
I20260619 02:53:23.538434 32057 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:23.538555 32057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 02:53:23 dist-test-slave-9mr0 krb5kdc[30845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837603, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for HTTP/127.18.134.20@KRBTEST.COM
I20260619 02:53:23.562758 32057 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:53:23.565542 32082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:23.567065 32057 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:53:23.567230 32057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "1cd4a86f7f0c4de8a71d50e53f14e249"
format_stamp: "Formatted at 2026-06-19 02:53:20 on dist-test-slave-9mr0"
server_key: "ec690fc916b8680095dac342e19f84b8"
server_key_iv: "8404e587d51bff49e9f404c9bfc9230a"
server_key_version: "kuduclusterkey@0"
I20260619 02:53:23.568209 32057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:23.586730 32057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:23.592109 32057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:23.592420 32057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:23.593526 32057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:53:23.594879 32057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:53:23.594965 32057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:23.595063 32057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:53:23.595127 32057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:23.657434 32057 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:45787
I20260619 02:53:23.657485 32196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:45787 every 8 connection(s)
I20260619 02:53:23.659098 32057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:53:23.668377 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 32057
I20260619 02:53:23.669102 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.134.20:36465/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:42699
--local_ip_for_outbound_sockets=127.18.134.3
--tserver_master_addrs=127.18.134.62:34193
--webserver_port=40823
--webserver_interface=127.18.134.3
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:36219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.134.20:36465/kms with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:53:23 dist-test-slave-9mr0 krb5kdc[30845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837603, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:53:23.705478 32200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:23.673410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:37589 (local address 127.18.134.62:34193)
0619 02:53:23.686812 (+ 13402us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:23.686818 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:23.686845 (+ 27us) server_negotiation.cc:407] Connection header received
0619 02:53:23.686915 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:23.686921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:23.687006 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:23.687135 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:23.689116 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:23.690179 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:23.692528 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:23.693128 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:23.693216 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:23.697728 (+ 4512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:23.697736 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:23.697741 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:23.697785 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:23.701557 (+ 3772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:23.702573 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:23.702580 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:23.702583 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:23.702764 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:23.703333 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:23.703338 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:23.703341 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:23.703720 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:23.703859 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:23.704885 (+ 1026us) server_negotiation.cc:299] Negotiation successful
0619 02:53:23.705117 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13193,"thread_start_us":13075,"threads_started":1}
I20260619 02:53:23.708614 32199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:23.672743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:34193 (local address 127.18.134.2:37589)
0619 02:53:23.677527 (+ 4784us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:23.677584 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:23.680179 (+ 2595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:23.687481 (+ 7302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:23.687495 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:23.688100 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:53:23.688905 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:23.688930 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:23.690400 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:23.690404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:23.692355 (+ 1951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:23.692371 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:23.693552 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:23.693559 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:23.693831 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:23.694611 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:53:23.694671 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:53:23.697431 (+ 2760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:53:23.701746 (+ 4315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:23.701755 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:23.701775 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:23.702421 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:23.702939 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:23.702944 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:23.702947 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:23.703196 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:23.703968 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:53:23.703976 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:53:23.704437 (+ 461us) client_negotiation.cc:769] Sending connection context
0619 02:53:23.706455 (+ 2018us) client_negotiation.cc:240] Negotiation successful
0619 02:53:23.707178 (+ 723us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4453,"thread_start_us":4169,"threads_started":1}
I20260619 02:53:23.710616 32197 heartbeater.cc:347] Connected to a master server at 127.18.134.62:34193
I20260619 02:53:23.710968 32197 heartbeater.cc:464] Registering TS with master...
I20260619 02:53:23.711826 32197 heartbeater.cc:511] Master 127.18.134.62:34193 requested a full tablet report, sending...
I20260619 02:53:23.713788 31856 ts_manager.cc:194] Registered new tserver with Master: 1cd4a86f7f0c4de8a71d50e53f14e249 (127.18.134.2:45787)
I20260619 02:53:23.715948 31856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.2@KRBTEST.COM'} at 127.18.134.2:37589
Jun 19 02:53:23 dist-test-slave-9mr0 krb5kdc[30845](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781837603, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:24.080142 32052 heartbeater.cc:464] Registering TS with master...
I20260619 02:53:24.080637 32052 heartbeater.cc:511] Master 127.18.134.62:34193 requested a full tablet report, sending...
I20260619 02:53:24.081841 31856 ts_manager.cc:194] Registered new tserver with Master: 4f182f97d3c24d2e8995592610ae5943 (127.18.134.1:42525)
I20260619 02:53:24.082974 31856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.1@KRBTEST.COM'} at 127.18.134.1:52065
W20260619 02:53:24.110482 32201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:24.110898 32201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:24.111063 32201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:53:24.122478 32201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:24.122651 32201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:24.122895 32201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:53:24.137042 32201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:36219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.134.20:36465/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:42699
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=40823
--tserver_master_addrs=127.18.134.62:34193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:24.139042 32201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:24.140801 32201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:53:24.157150 32207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
Jun 19 02:53:24 dist-test-slave-9mr0 krb5kdc[30845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837603, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for HTTP/127.18.134.20@KRBTEST.COM
W20260619 02:53:24.158845 32208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:24.166507 32210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:24.168938 32201 server_base.cc:1034] running on GCE node
I20260619 02:53:24.169759 32201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:24.175304 32201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:24.176584 32201 hybrid_clock.cc:648] HybridClock initialized: now 1781837604176479 us; error 99 us; skew 500 ppm
I20260619 02:53:24.177035 32201 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:24 dist-test-slave-9mr0 krb5kdc[30845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837604, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:24.182082 32201 init.cc:377] Logged in from keytab as kudu/127.18.134.3@KRBTEST.COM (short username kudu)
I20260619 02:53:24.195943 32201 webserver.cc:460] Webserver started at http://127.18.134.3:40823/ using document root <none> and password file <none>
I20260619 02:53:24.196731 32201 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:24.196879 32201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 02:53:24 dist-test-slave-9mr0 krb5kdc[30845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837604, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for HTTP/127.18.134.20@KRBTEST.COM
I20260619 02:53:24.229512 32201 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 02:53:24.233109 32218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:24.234903 32201 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260619 02:53:24.235077 32201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "744815233d3245b582b01585c7f478a9"
format_stamp: "Formatted at 2026-06-19 02:53:21 on dist-test-slave-9mr0"
server_key: "d6a12027c84b6fb01c9656f0dc7034a0"
server_key_iv: "da330b1e7d18c1f07bab54be6d42fbe0"
server_key_version: "kuduclusterkey@0"
I20260619 02:53:24.236438 32201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:24.255875 32201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:24.259897 32201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:24.260164 32201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:24.261168 32201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:53:24.262593 32201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:53:24.262710 32201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260619 02:53:24.262818 32201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:53:24.262890 32201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:24.313364 32201 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:42699
I20260619 02:53:24.313443 32330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:42699 every 8 connection(s)
I20260619 02:53:24.315286 32201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:53:24.319977 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 32201
Jun 19 02:53:24 dist-test-slave-9mr0 krb5kdc[30845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837604, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:53:24.352432 32336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:24.321471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45590 (local address 127.18.134.62:34193)
0619 02:53:24.331812 (+ 10341us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:24.331817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:24.331833 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:53:24.331879 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:24.331883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:24.331960 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:24.332056 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:24.334106 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.335367 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.337402 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.338276 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.338385 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:24.342015 (+ 3630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:24.342022 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:24.342027 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:24.342074 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:24.348377 (+ 6303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:24.349376 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:24.349382 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:24.349386 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:24.349539 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:24.349952 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:24.349957 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:24.349961 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:24.350316 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:24.350589 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:24.351829 (+ 1240us) server_negotiation.cc:299] Negotiation successful
0619 02:53:24.352060 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10235,"thread_start_us":10098,"threads_started":1}
I20260619 02:53:24.355376 32334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:24.319405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:48325 (local address 127.18.134.62:34193)
0619 02:53:24.330222 (+ 10817us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:24.330229 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:24.330259 (+ 30us) server_negotiation.cc:407] Connection header received
0619 02:53:24.330326 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:24.330331 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:24.330430 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:24.330539 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:24.333602 (+ 3063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.334698 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.336919 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.337482 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.337586 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:24.341607 (+ 4021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:24.341655 (+ 48us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:24.341660 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:24.341702 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:24.345549 (+ 3847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:24.351582 (+ 6033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:24.351588 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:24.351591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:24.351726 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:24.352293 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:24.352298 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:24.352301 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:24.352627 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:24.352760 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:24.354807 (+ 2047us) server_negotiation.cc:299] Negotiation successful
0619 02:53:24.355018 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":823,"server-negotiator.queue_time_us":10683,"thread_start_us":10574,"threads_started":1}
I20260619 02:53:24.356606 32333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:24.318824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:34193 (local address 127.18.134.3:48325)
0619 02:53:24.326875 (+ 8051us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:24.326931 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:24.329466 (+ 2535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:24.332397 (+ 2931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:24.332407 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:24.332822 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:53:24.333421 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:24.333439 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.334878 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.334884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:24.336748 (+ 1864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:24.336774 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.337766 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.337772 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:24.338076 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:24.338886 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:53:24.338920 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:53:24.341314 (+ 2394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:53:24.350736 (+ 9422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:24.350747 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:24.350765 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:24.351410 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:24.351874 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:24.351879 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:24.351883 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:24.352150 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:24.352832 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:53:24.352841 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:53:24.353273 (+ 432us) client_negotiation.cc:769] Sending connection context
0619 02:53:24.353559 (+ 286us) client_negotiation.cc:240] Negotiation successful
0619 02:53:24.354100 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7703,"thread_start_us":7485,"threads_started":1}
I20260619 02:53:24.358063 32331 heartbeater.cc:347] Connected to a master server at 127.18.134.62:34193
I20260619 02:53:24.358374 32331 heartbeater.cc:464] Registering TS with master...
I20260619 02:53:24.359232 32331 heartbeater.cc:511] Master 127.18.134.62:34193 requested a full tablet report, sending...
I20260619 02:53:24.360893 31856 ts_manager.cc:194] Registered new tserver with Master: 744815233d3245b582b01585c7f478a9 (127.18.134.3:42699)
I20260619 02:53:24.362064 31856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.3@KRBTEST.COM'} at 127.18.134.3:48325
I20260619 02:53:24.368199 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:53:24.389513 32334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:24.371927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45592 (local address 127.18.134.62:34193)
0619 02:53:24.372109 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:24.372115 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:24.372610 (+ 495us) server_negotiation.cc:407] Connection header received
0619 02:53:24.372782 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:24.372788 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:24.372870 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:24.372983 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:24.374942 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.376066 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.378027 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.378696 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.378839 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:24.382139 (+ 3300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:24.382146 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:24.382151 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:24.382192 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:24.385890 (+ 3698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:24.386886 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:24.386892 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:24.386895 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:24.387054 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:24.387500 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:24.387505 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:24.387508 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:24.387863 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:24.388102 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:24.388861 (+ 759us) server_negotiation.cc:299] Negotiation successful
0619 02:53:24.389126 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 02:53:24.395951 31856 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45592:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:53:24.406822 31856 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Jun 19 02:53:24 dist-test-slave-9mr0 krb5kdc[30845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837603, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for HTTP/127.18.134.20@KRBTEST.COM
I20260619 02:53:24.486380 32348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:24.445257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:45787 (local address 127.0.0.1:40672)
0619 02:53:24.469911 (+ 24654us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:24.469970 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:24.470165 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:24.472780 (+ 2615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:24.472789 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:24.472828 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:53:24.473198 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:24.473210 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.481087 (+ 7877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.481093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:24.483173 (+ 2080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:24.483189 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.485405 (+ 2216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.485410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:24.485606 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:24.485668 (+ 62us) client_negotiation.cc:769] Sending connection context
0619 02:53:24.485802 (+ 134us) client_negotiation.cc:240] Negotiation successful
0619 02:53:24.486011 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24348,"thread_start_us":25830,"threads_started":1}
I20260619 02:53:24.489924 32347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:24.445710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:42699 (local address 127.0.0.1:35656)
0619 02:53:24.471323 (+ 25613us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:24.471344 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:24.473413 (+ 2069us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:24.474344 (+ 931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:24.474349 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:24.474388 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:53:24.474758 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:24.474769 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.476786 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.476792 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:24.479206 (+ 2414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:24.479220 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.489109 (+ 9889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.489117 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:24.489301 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:24.489336 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 02:53:24.489436 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 02:53:24.489583 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25499,"thread_start_us":24658,"threads_started":1}
I20260619 02:53:24.490605 32350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:24.446272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35656 (local address 127.18.134.3:42699)
0619 02:53:24.469101 (+ 22829us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:24.469112 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:24.471452 (+ 2340us) server_negotiation.cc:407] Connection header received
0619 02:53:24.473618 (+ 2166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:24.473626 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:24.473856 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:24.474012 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:53:24.474959 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.476589 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.487147 (+ 10558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.488743 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.488855 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:24.489078 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:24.489543 (+ 465us) server_negotiation.cc:299] Negotiation successful
0619 02:53:24.489713 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22616,"thread_start_us":22850,"threads_started":1}
I20260619 02:53:24.492463 32349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:24.445840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40672 (local address 127.18.134.2:45787)
0619 02:53:24.471849 (+ 26009us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:24.471858 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:24.471884 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:53:24.471980 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:24.471988 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:24.472265 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:24.472476 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:53:24.479405 (+ 6929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.480926 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.483419 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.484892 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.485028 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:24.485240 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:24.491936 (+ 6696us) server_negotiation.cc:299] Negotiation successful
0619 02:53:24.492123 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25845,"thread_start_us":33334,"threads_started":1}
I20260619 02:53:24.494755 32266 tablet_service.cc:1467] Processing CreateTablet for tablet 27dc0afc166c46b98a4219012022f781 (DEFAULT_TABLE table=test-table [id=4fa51f1d817d41cb8ab1f35176c7ff8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:53:24.496345 32266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27dc0afc166c46b98a4219012022f781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:53:24.499774 32348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:24.480466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:42525 (local address 127.0.0.1:38272)
0619 02:53:24.486868 (+ 6402us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:24.486885 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:24.486979 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:24.493119 (+ 6140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:24.493124 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:24.493154 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:53:24.493314 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:24.493325 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.495263 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.495268 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:24.497013 (+ 1745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:24.497037 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.499051 (+ 2014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.499056 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:24.499257 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:24.499294 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 02:53:24.499388 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 02:53:24.499521 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6114,"threads_started":1}
I20260619 02:53:24.500491 32351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:24.471927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38272 (local address 127.18.134.1:42525)
0619 02:53:24.490989 (+ 19062us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:24.490999 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:24.491033 (+ 34us) server_negotiation.cc:407] Connection header received
0619 02:53:24.491133 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:24.491141 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:24.491441 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:24.491636 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:53:24.493520 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.495110 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.497204 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.498568 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.498738 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:24.498957 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:24.499964 (+ 1007us) server_negotiation.cc:299] Negotiation successful
0619 02:53:24.500123 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18454,"threads_started":1}
I20260619 02:53:24.507055 31987 tablet_service.cc:1467] Processing CreateTablet for tablet 27dc0afc166c46b98a4219012022f781 (DEFAULT_TABLE table=test-table [id=4fa51f1d817d41cb8ab1f35176c7ff8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:53:24.508654 31987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27dc0afc166c46b98a4219012022f781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:53:24.518252 32353 tablet_bootstrap.cc:492] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9: Bootstrap starting.
I20260619 02:53:24.521477 32130 tablet_service.cc:1467] Processing CreateTablet for tablet 27dc0afc166c46b98a4219012022f781 (DEFAULT_TABLE table=test-table [id=4fa51f1d817d41cb8ab1f35176c7ff8d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:53:24.523276 32130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27dc0afc166c46b98a4219012022f781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:53:24.524251 32353 tablet_bootstrap.cc:654] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9: Neither blocks nor log segments found. Creating new log.
I20260619 02:53:24.525557 32353 log.cc:824] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9: Log is configured to *not* fsync() on all Append() calls
I20260619 02:53:24.534785 32353 tablet_bootstrap.cc:492] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9: No bootstrap required, opened a new log
I20260619 02:53:24.535199 32353 ts_tablet_manager.cc:1397] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.004s
I20260619 02:53:24.545233 32353 raft_consensus.cc:348] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd4a86f7f0c4de8a71d50e53f14e249" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45787 } } peers { permanent_uuid: "4f182f97d3c24d2e8995592610ae5943" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 42525 } } peers { permanent_uuid: "744815233d3245b582b01585c7f478a9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42699 } }
I20260619 02:53:24.546468 32355 tablet_bootstrap.cc:492] T 27dc0afc166c46b98a4219012022f781 P 1cd4a86f7f0c4de8a71d50e53f14e249: Bootstrap starting.
I20260619 02:53:24.550267 32355 tablet_bootstrap.cc:654] T 27dc0afc166c46b98a4219012022f781 P 1cd4a86f7f0c4de8a71d50e53f14e249: Neither blocks nor log segments found. Creating new log.
I20260619 02:53:24.550518 32353 raft_consensus.cc:374] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:53:24.550719 32353 raft_consensus.cc:729] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 744815233d3245b582b01585c7f478a9, State: Initialized, Role: FOLLOWER
I20260619 02:53:24.551662 32353 consensus_queue.cc:260] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd4a86f7f0c4de8a71d50e53f14e249" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45787 } } peers { permanent_uuid: "4f182f97d3c24d2e8995592610ae5943" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 42525 } } peers { permanent_uuid: "744815233d3245b582b01585c7f478a9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42699 } }
I20260619 02:53:24.553484 32355 log.cc:824] T 27dc0afc166c46b98a4219012022f781 P 1cd4a86f7f0c4de8a71d50e53f14e249: Log is configured to *not* fsync() on all Append() calls
I20260619 02:53:24.554739 32356 tablet_bootstrap.cc:492] T 27dc0afc166c46b98a4219012022f781 P 4f182f97d3c24d2e8995592610ae5943: Bootstrap starting.
I20260619 02:53:24.559087 32356 tablet_bootstrap.cc:654] T 27dc0afc166c46b98a4219012022f781 P 4f182f97d3c24d2e8995592610ae5943: Neither blocks nor log segments found. Creating new log.
I20260619 02:53:24.560307 32356 log.cc:824] T 27dc0afc166c46b98a4219012022f781 P 4f182f97d3c24d2e8995592610ae5943: Log is configured to *not* fsync() on all Append() calls
I20260619 02:53:24.563683 32355 tablet_bootstrap.cc:492] T 27dc0afc166c46b98a4219012022f781 P 1cd4a86f7f0c4de8a71d50e53f14e249: No bootstrap required, opened a new log
I20260619 02:53:24.564608 32331 heartbeater.cc:503] Master 127.18.134.62:34193 was elected leader, sending a full tablet report...
I20260619 02:53:24.565418 32353 ts_tablet_manager.cc:1428] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9: Time spent starting tablet: real 0.030s user 0.015s sys 0.001s
I20260619 02:53:24.566016 32355 ts_tablet_manager.cc:1397] T 27dc0afc166c46b98a4219012022f781 P 1cd4a86f7f0c4de8a71d50e53f14e249: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.004s
W20260619 02:53:24.577839 32332 tablet.cc:2369] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:53:24.582619 32355 raft_consensus.cc:348] T 27dc0afc166c46b98a4219012022f781 P 1cd4a86f7f0c4de8a71d50e53f14e249 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd4a86f7f0c4de8a71d50e53f14e249" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45787 } } peers { permanent_uuid: "4f182f97d3c24d2e8995592610ae5943" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 42525 } } peers { permanent_uuid: "744815233d3245b582b01585c7f478a9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42699 } }
I20260619 02:53:24.583230 32355 raft_consensus.cc:374] T 27dc0afc166c46b98a4219012022f781 P 1cd4a86f7f0c4de8a71d50e53f14e249 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:53:24.583401 32355 raft_consensus.cc:729] T 27dc0afc166c46b98a4219012022f781 P 1cd4a86f7f0c4de8a71d50e53f14e249 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cd4a86f7f0c4de8a71d50e53f14e249, State: Initialized, Role: FOLLOWER
I20260619 02:53:24.584185 32355 consensus_queue.cc:260] T 27dc0afc166c46b98a4219012022f781 P 1cd4a86f7f0c4de8a71d50e53f14e249 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd4a86f7f0c4de8a71d50e53f14e249" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45787 } } peers { permanent_uuid: "4f182f97d3c24d2e8995592610ae5943" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 42525 } } peers { permanent_uuid: "744815233d3245b582b01585c7f478a9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42699 } }
I20260619 02:53:24.592979 32355 ts_tablet_manager.cc:1428] T 27dc0afc166c46b98a4219012022f781 P 1cd4a86f7f0c4de8a71d50e53f14e249: Time spent starting tablet: real 0.021s user 0.019s sys 0.000s
I20260619 02:53:24.593401 32197 heartbeater.cc:503] Master 127.18.134.62:34193 was elected leader, sending a full tablet report...
I20260619 02:53:24.595816 32356 tablet_bootstrap.cc:492] T 27dc0afc166c46b98a4219012022f781 P 4f182f97d3c24d2e8995592610ae5943: No bootstrap required, opened a new log
I20260619 02:53:24.596181 32356 ts_tablet_manager.cc:1397] T 27dc0afc166c46b98a4219012022f781 P 4f182f97d3c24d2e8995592610ae5943: Time spent bootstrapping tablet: real 0.042s user 0.020s sys 0.000s
I20260619 02:53:24.614050 32356 raft_consensus.cc:348] T 27dc0afc166c46b98a4219012022f781 P 4f182f97d3c24d2e8995592610ae5943 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd4a86f7f0c4de8a71d50e53f14e249" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45787 } } peers { permanent_uuid: "4f182f97d3c24d2e8995592610ae5943" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 42525 } } peers { permanent_uuid: "744815233d3245b582b01585c7f478a9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42699 } }
I20260619 02:53:24.614707 32356 raft_consensus.cc:374] T 27dc0afc166c46b98a4219012022f781 P 4f182f97d3c24d2e8995592610ae5943 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:53:24.614862 32356 raft_consensus.cc:729] T 27dc0afc166c46b98a4219012022f781 P 4f182f97d3c24d2e8995592610ae5943 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f182f97d3c24d2e8995592610ae5943, State: Initialized, Role: FOLLOWER
I20260619 02:53:24.615631 32356 consensus_queue.cc:260] T 27dc0afc166c46b98a4219012022f781 P 4f182f97d3c24d2e8995592610ae5943 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd4a86f7f0c4de8a71d50e53f14e249" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45787 } } peers { permanent_uuid: "4f182f97d3c24d2e8995592610ae5943" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 42525 } } peers { permanent_uuid: "744815233d3245b582b01585c7f478a9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42699 } }
I20260619 02:53:24.617267 32052 heartbeater.cc:503] Master 127.18.134.62:34193 was elected leader, sending a full tablet report...
I20260619 02:53:24.619750 32356 ts_tablet_manager.cc:1428] T 27dc0afc166c46b98a4219012022f781 P 4f182f97d3c24d2e8995592610ae5943: Time spent starting tablet: real 0.023s user 0.013s sys 0.000s
I20260619 02:53:24.648533 32357 raft_consensus.cc:482] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:53:24.648851 32357 raft_consensus.cc:504] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd4a86f7f0c4de8a71d50e53f14e249" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45787 } } peers { permanent_uuid: "4f182f97d3c24d2e8995592610ae5943" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 42525 } } peers { permanent_uuid: "744815233d3245b582b01585c7f478a9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42699 } }
I20260619 02:53:24.650695 32357 leader_election.cc:288] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cd4a86f7f0c4de8a71d50e53f14e249 (127.18.134.2:45787), 4f182f97d3c24d2e8995592610ae5943 (127.18.134.1:42525)
I20260619 02:53:24.659597 32360 raft_consensus.cc:482] T 27dc0afc166c46b98a4219012022f781 P 1cd4a86f7f0c4de8a71d50e53f14e249 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:53:24.659920 32360 raft_consensus.cc:504] T 27dc0afc166c46b98a4219012022f781 P 1cd4a86f7f0c4de8a71d50e53f14e249 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd4a86f7f0c4de8a71d50e53f14e249" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45787 } } peers { permanent_uuid: "4f182f97d3c24d2e8995592610ae5943" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 42525 } } peers { permanent_uuid: "744815233d3245b582b01585c7f478a9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42699 } }
I20260619 02:53:24.661811 32360 leader_election.cc:288] T 27dc0afc166c46b98a4219012022f781 P 1cd4a86f7f0c4de8a71d50e53f14e249 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f182f97d3c24d2e8995592610ae5943 (127.18.134.1:42525), 744815233d3245b582b01585c7f478a9 (127.18.134.3:42699)
I20260619 02:53:24.665318 32333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:24.651433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:42525 (local address 127.18.134.3:51411)
0619 02:53:24.658784 (+ 7351us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:24.658803 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:24.658926 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:24.659433 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:24.659437 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:24.659462 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:53:24.659659 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:24.659670 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.661064 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.661069 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:24.662952 (+ 1883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:24.662966 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.664607 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.664613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:24.664788 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:24.664820 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 02:53:24.664905 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 02:53:24.665035 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7223,"thread_start_us":5266,"threads_started":1}
I20260619 02:53:24.665856 32351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:24.652367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:51411 (local address 127.18.134.1:42525)
0619 02:53:24.652722 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:24.652728 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:24.659032 (+ 6304us) server_negotiation.cc:407] Connection header received
0619 02:53:24.659119 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:24.659124 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:24.659209 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:24.659286 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:53:24.659828 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.660888 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.663122 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.664325 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.664461 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:24.664532 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:24.665503 (+ 971us) server_negotiation.cc:299] Negotiation successful
0619 02:53:24.665634 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237}
W20260619 02:53:24.667392 32198 tablet.cc:2369] T 27dc0afc166c46b98a4219012022f781 P 1cd4a86f7f0c4de8a71d50e53f14e249: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:53:24.672418 32007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27dc0afc166c46b98a4219012022f781" candidate_uuid: "744815233d3245b582b01585c7f478a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f182f97d3c24d2e8995592610ae5943" is_pre_election: true
I20260619 02:53:24.673005 32007 raft_consensus.cc:2454] T 27dc0afc166c46b98a4219012022f781 P 4f182f97d3c24d2e8995592610ae5943 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 744815233d3245b582b01585c7f478a9 in term 0.
I20260619 02:53:24.673947 32220 leader_election.cc:302] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f182f97d3c24d2e8995592610ae5943, 744815233d3245b582b01585c7f478a9; no voters:
I20260619 02:53:24.674795 32357 raft_consensus.cc:2781] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:53:24.674969 32357 raft_consensus.cc:482] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:53:24.675062 32357 raft_consensus.cc:3037] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:53:24.679823 32357 raft_consensus.cc:504] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd4a86f7f0c4de8a71d50e53f14e249" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45787 } } peers { permanent_uuid: "4f182f97d3c24d2e8995592610ae5943" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 42525 } } peers { permanent_uuid: "744815233d3245b582b01585c7f478a9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42699 } }
I20260619 02:53:24.680702 32357 leader_election.cc:288] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9 [CANDIDATE]: Term 1 election: Requested vote from peers 1cd4a86f7f0c4de8a71d50e53f14e249 (127.18.134.2:45787), 4f182f97d3c24d2e8995592610ae5943 (127.18.134.1:42525)
I20260619 02:53:24.681264 32351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:24.662796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:35425 (local address 127.18.134.1:42525)
0619 02:53:24.666054 (+ 3258us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:24.666060 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:24.671144 (+ 5084us) server_negotiation.cc:407] Connection header received
0619 02:53:24.671795 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:24.671801 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:24.671876 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:24.671934 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:53:24.672704 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.673877 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.678743 (+ 4866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.679953 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.680090 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:24.680161 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:24.680827 (+ 666us) server_negotiation.cc:299] Negotiation successful
0619 02:53:24.680964 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3166,"thread_start_us":8671,"threads_started":1}
I20260619 02:53:24.681293 32007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27dc0afc166c46b98a4219012022f781" candidate_uuid: "744815233d3245b582b01585c7f478a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f182f97d3c24d2e8995592610ae5943"
I20260619 02:53:24.681511 32007 raft_consensus.cc:3037] T 27dc0afc166c46b98a4219012022f781 P 4f182f97d3c24d2e8995592610ae5943 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:53:24.681913 32349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:24.652112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:52337 (local address 127.18.134.2:45787)
0619 02:53:24.652579 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:24.652585 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:24.657211 (+ 4626us) server_negotiation.cc:407] Connection header received
0619 02:53:24.657519 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:24.657525 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:24.657611 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:24.657685 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:53:24.658182 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.659205 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.673247 (+ 14042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.674454 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.674573 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:24.674663 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:24.681470 (+ 6807us) server_negotiation.cc:299] Negotiation successful
0619 02:53:24.681640 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":323}
I20260619 02:53:24.682452 32365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:24.662331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:42525 (local address 127.18.134.2:35425)
0619 02:53:24.671044 (+ 8713us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:24.671065 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:24.671256 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:24.672114 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:24.672119 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:24.672145 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:53:24.672534 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:24.672548 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.674064 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.674069 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:24.676201 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:24.676217 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.680364 (+ 4147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.680371 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:24.680582 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:24.680618 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 02:53:24.681441 (+ 823us) client_negotiation.cc:240] Negotiation successful
0619 02:53:24.682129 (+ 688us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8592,"mutex_wait_us":526,"threads_started":1}
I20260619 02:53:24.682512 32362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:24.651027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:45787 (local address 127.18.134.3:52337)
0619 02:53:24.657102 (+ 6075us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:24.657123 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:24.657337 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:24.657718 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:24.657722 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:24.657747 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:53:24.658037 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:24.658050 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.670927 (+ 12877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.670933 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:24.673025 (+ 2092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:24.673041 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.681116 (+ 8075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.681122 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:24.681289 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:24.681322 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 02:53:24.682086 (+ 764us) client_negotiation.cc:240] Negotiation successful
0619 02:53:24.682232 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5850}
I20260619 02:53:24.684568 32007 raft_consensus.cc:2454] T 27dc0afc166c46b98a4219012022f781 P 4f182f97d3c24d2e8995592610ae5943 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 744815233d3245b582b01585c7f478a9 in term 1.
I20260619 02:53:24.685645 32150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27dc0afc166c46b98a4219012022f781" candidate_uuid: "744815233d3245b582b01585c7f478a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cd4a86f7f0c4de8a71d50e53f14e249" is_pre_election: true
I20260619 02:53:24.686054 32150 raft_consensus.cc:2454] T 27dc0afc166c46b98a4219012022f781 P 1cd4a86f7f0c4de8a71d50e53f14e249 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 744815233d3245b582b01585c7f478a9 in term 0.
I20260619 02:53:24.686585 32149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27dc0afc166c46b98a4219012022f781" candidate_uuid: "744815233d3245b582b01585c7f478a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cd4a86f7f0c4de8a71d50e53f14e249"
I20260619 02:53:24.686865 32149 raft_consensus.cc:3037] T 27dc0afc166c46b98a4219012022f781 P 1cd4a86f7f0c4de8a71d50e53f14e249 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:53:24.690016 32149 raft_consensus.cc:2454] T 27dc0afc166c46b98a4219012022f781 P 1cd4a86f7f0c4de8a71d50e53f14e249 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 744815233d3245b582b01585c7f478a9 in term 1.
I20260619 02:53:24.691097 32220 leader_election.cc:302] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cd4a86f7f0c4de8a71d50e53f14e249, 744815233d3245b582b01585c7f478a9; no voters:
I20260619 02:53:24.691671 32350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:24.663590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:58919 (local address 127.18.134.3:42699)
0619 02:53:24.663766 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:24.663772 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:24.682755 (+ 18983us) server_negotiation.cc:407] Connection header received
0619 02:53:24.683044 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:24.683049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:24.683190 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:24.683342 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:53:24.684234 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.685820 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.688966 (+ 3146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.690591 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.690748 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:24.690814 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:24.691333 (+ 519us) server_negotiation.cc:299] Negotiation successful
0619 02:53:24.691452 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:53:24.691908 32357 raft_consensus.cc:2781] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:53:24.692232 32365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:24.663274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:42699 (local address 127.18.134.2:58919)
0619 02:53:24.682668 (+ 19394us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:24.682687 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:24.682848 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:24.683626 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:24.683631 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:24.683694 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:53:24.683973 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:24.684016 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.686058 (+ 2042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.686064 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:24.688750 (+ 2686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:24.688764 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.690953 (+ 2189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.690959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:24.691155 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:24.691190 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 02:53:24.691813 (+ 623us) client_negotiation.cc:240] Negotiation successful
0619 02:53:24.691971 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19267,"thread_start_us":7567,"threads_started":1}
I20260619 02:53:24.693174 32286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27dc0afc166c46b98a4219012022f781" candidate_uuid: "1cd4a86f7f0c4de8a71d50e53f14e249" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "744815233d3245b582b01585c7f478a9" is_pre_election: true
I20260619 02:53:24.696877 32007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27dc0afc166c46b98a4219012022f781" candidate_uuid: "1cd4a86f7f0c4de8a71d50e53f14e249" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f182f97d3c24d2e8995592610ae5943" is_pre_election: true
I20260619 02:53:24.697165 32007 raft_consensus.cc:2379] T 27dc0afc166c46b98a4219012022f781 P 4f182f97d3c24d2e8995592610ae5943 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1cd4a86f7f0c4de8a71d50e53f14e249 in current term 1: Already voted for candidate 744815233d3245b582b01585c7f478a9 in this term.
I20260619 02:53:24.700912 32357 raft_consensus.cc:686] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9 [term 1 LEADER]: Becoming Leader. State: Replica: 744815233d3245b582b01585c7f478a9, State: Running, Role: LEADER
I20260619 02:53:24.701737 32357 consensus_queue.cc:237] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd4a86f7f0c4de8a71d50e53f14e249" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45787 } } peers { permanent_uuid: "4f182f97d3c24d2e8995592610ae5943" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 42525 } } peers { permanent_uuid: "744815233d3245b582b01585c7f478a9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42699 } }
I20260619 02:53:24.707111 32086 leader_election.cc:302] T 27dc0afc166c46b98a4219012022f781 P 1cd4a86f7f0c4de8a71d50e53f14e249 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1cd4a86f7f0c4de8a71d50e53f14e249; no voters: 4f182f97d3c24d2e8995592610ae5943, 744815233d3245b582b01585c7f478a9
I20260619 02:53:24.707918 32360 raft_consensus.cc:2726] T 27dc0afc166c46b98a4219012022f781 P 1cd4a86f7f0c4de8a71d50e53f14e249 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 02:53:24.707989 31856 catalog_manager.cc:5495] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 744815233d3245b582b01585c7f478a9 (127.18.134.3). New cstate: current_term: 1 leader_uuid: "744815233d3245b582b01585c7f478a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cd4a86f7f0c4de8a71d50e53f14e249" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f182f97d3c24d2e8995592610ae5943" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 42525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "744815233d3245b582b01585c7f478a9" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42699 } health_report { overall_health: HEALTHY } } }
I20260619 02:53:24.752998 32350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:24.745115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35668 (local address 127.18.134.3:42699)
0619 02:53:24.745284 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:24.745291 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:24.745598 (+ 307us) server_negotiation.cc:407] Connection header received
0619 02:53:24.745798 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:24.745804 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:24.745918 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:24.746066 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:53:24.746688 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.747775 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.749866 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.750355 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.750457 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:24.751149 (+ 692us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:53:24.751880 (+ 731us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:53:24.752020 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:24.752602 (+ 582us) server_negotiation.cc:299] Negotiation successful
0619 02:53:24.752686 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 02:53:24.765913 32007 raft_consensus.cc:1261] T 27dc0afc166c46b98a4219012022f781 P 4f182f97d3c24d2e8995592610ae5943 [term 1 FOLLOWER]: Refusing update from remote peer 744815233d3245b582b01585c7f478a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260619 02:53:24.766219 32053 tablet.cc:2369] T 27dc0afc166c46b98a4219012022f781 P 4f182f97d3c24d2e8995592610ae5943: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:53:24.767681 32149 raft_consensus.cc:1261] T 27dc0afc166c46b98a4219012022f781 P 1cd4a86f7f0c4de8a71d50e53f14e249 [term 1 FOLLOWER]: Refusing update from remote peer 744815233d3245b582b01585c7f478a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:53:24.767706 32357 consensus_queue.cc:1035] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f182f97d3c24d2e8995592610ae5943" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 42525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:53:24.768476 32357 consensus_queue.cc:1035] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cd4a86f7f0c4de8a71d50e53f14e249" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:53:24.785832 32369 mvcc.cc:205] Tried to move back new op lower bound from 7298406829107777536 to 7298406828863307776. Current Snapshot: MvccSnapshot[applied={T|T < 7298406829107777536}]
I20260619 02:53:24.812650 31856 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:53:24.813001 31856 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:53:24.818369 31856 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 428a8cb6b5a844fdad3805c9772e67a0: Sending DeleteTablet for 3 replicas of tablet 27dc0afc166c46b98a4219012022f781
I20260619 02:53:24.819506 32130 tablet_service.cc:1514] Processing DeleteTablet for tablet 27dc0afc166c46b98a4219012022f781 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:53:24 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:40672
I20260619 02:53:24.819826 31987 tablet_service.cc:1514] Processing DeleteTablet for tablet 27dc0afc166c46b98a4219012022f781 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:53:24 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:38272
I20260619 02:53:24.820346 32266 tablet_service.cc:1514] Processing DeleteTablet for tablet 27dc0afc166c46b98a4219012022f781 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:53:24 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:35656
I20260619 02:53:24.832124 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 31893
I20260619 02:53:24.832183 32379 tablet_replica.cc:330] T 27dc0afc166c46b98a4219012022f781 P 4f182f97d3c24d2e8995592610ae5943: stopping tablet replica
I20260619 02:53:24.832597 32379 raft_consensus.cc:2229] T 27dc0afc166c46b98a4219012022f781 P 4f182f97d3c24d2e8995592610ae5943 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:53:24.833128 32379 raft_consensus.cc:2258] T 27dc0afc166c46b98a4219012022f781 P 4f182f97d3c24d2e8995592610ae5943 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:53:24.839409 32378 tablet_replica.cc:330] T 27dc0afc166c46b98a4219012022f781 P 1cd4a86f7f0c4de8a71d50e53f14e249: stopping tablet replica
I20260619 02:53:24.840016 32378 raft_consensus.cc:2229] T 27dc0afc166c46b98a4219012022f781 P 1cd4a86f7f0c4de8a71d50e53f14e249 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:53:24.840569 32378 raft_consensus.cc:2258] T 27dc0afc166c46b98a4219012022f781 P 1cd4a86f7f0c4de8a71d50e53f14e249 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:53:24.843575 32379 ts_tablet_manager.cc:1905] T 27dc0afc166c46b98a4219012022f781 P 4f182f97d3c24d2e8995592610ae5943: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:53:24.845273 32378 ts_tablet_manager.cc:1905] T 27dc0afc166c46b98a4219012022f781 P 1cd4a86f7f0c4de8a71d50e53f14e249: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:53:24.850739 32379 ts_tablet_manager.cc:1918] T 27dc0afc166c46b98a4219012022f781 P 4f182f97d3c24d2e8995592610ae5943: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:53:24.850920 32379 log.cc:1196] T 27dc0afc166c46b98a4219012022f781 P 4f182f97d3c24d2e8995592610ae5943: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-0/wal/wals/27dc0afc166c46b98a4219012022f781
I20260619 02:53:24.851495 32379 ts_tablet_manager.cc:1939] T 27dc0afc166c46b98a4219012022f781 P 4f182f97d3c24d2e8995592610ae5943: Deleting consensus metadata
I20260619 02:53:24.853466 31841 catalog_manager.cc:4841] TS 4f182f97d3c24d2e8995592610ae5943 (127.18.134.1:42525): tablet 27dc0afc166c46b98a4219012022f781 (table test-table [id=4fa51f1d817d41cb8ab1f35176c7ff8d]) successfully deleted
I20260619 02:53:24.855970 32351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:24.834162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38282 (local address 127.18.134.1:42525)
0619 02:53:24.834341 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:24.834347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:24.834368 (+ 21us) server_negotiation.cc:407] Connection header received
0619 02:53:24.834432 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:24.834437 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:24.834528 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:24.834624 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:24.836559 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.837548 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.840908 (+ 3360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:24.841559 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:24.841682 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:24.846307 (+ 4625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:24.846322 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:24.846339 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:24.846381 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:24.850244 (+ 3863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:24.851509 (+ 1265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:24.851518 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:24.851522 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:24.851699 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:24.852678 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:24.852683 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:24.852687 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:24.853145 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:24.853918 (+ 773us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:24.855379 (+ 1461us) server_negotiation.cc:299] Negotiation successful
0619 02:53:24.855607 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 02:53:24.856700 32047 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:53:24.860783 32378 ts_tablet_manager.cc:1918] T 27dc0afc166c46b98a4219012022f781 P 1cd4a86f7f0c4de8a71d50e53f14e249: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:53:24.868006 32380 tablet_replica.cc:330] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9: stopping tablet replica
I20260619 02:53:24.868533 32380 raft_consensus.cc:2229] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:53:24.868897 32378 log.cc:1196] T 27dc0afc166c46b98a4219012022f781 P 1cd4a86f7f0c4de8a71d50e53f14e249: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-1/wal/wals/27dc0afc166c46b98a4219012022f781
I20260619 02:53:24.869563 32378 ts_tablet_manager.cc:1939] T 27dc0afc166c46b98a4219012022f781 P 1cd4a86f7f0c4de8a71d50e53f14e249: Deleting consensus metadata
I20260619 02:53:24.869810 32380 raft_consensus.cc:2258] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:53:24.872978 31841 catalog_manager.cc:4841] TS 1cd4a86f7f0c4de8a71d50e53f14e249 (127.18.134.2:45787): tablet 27dc0afc166c46b98a4219012022f781 (table test-table [id=4fa51f1d817d41cb8ab1f35176c7ff8d]) successfully deleted
I20260619 02:53:24.873071 32380 ts_tablet_manager.cc:1905] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:53:24.880303 32380 ts_tablet_manager.cc:1918] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:53:24.880488 32380 log.cc:1196] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781837410515220-18968-0/minicluster-data/ts-2/wal/wals/27dc0afc166c46b98a4219012022f781
I20260619 02:53:24.881093 32380 ts_tablet_manager.cc:1939] T 27dc0afc166c46b98a4219012022f781 P 744815233d3245b582b01585c7f478a9: Deleting consensus metadata
I20260619 02:53:24.882962 31843 catalog_manager.cc:4841] TS 744815233d3245b582b01585c7f478a9 (127.18.134.3:42699): tablet 27dc0afc166c46b98a4219012022f781 (table test-table [id=4fa51f1d817d41cb8ab1f35176c7ff8d]) successfully deleted
I20260619 02:53:25.034955 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 31893
I20260619 02:53:25.055404 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 32057
I20260619 02:53:25.074034 32349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:25.056791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40678 (local address 127.18.134.2:45787)
0619 02:53:25.056956 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:25.056961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:25.056979 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:53:25.057037 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:25.057042 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:25.057120 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:25.057205 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:25.059300 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:25.060368 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:25.062259 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:25.062896 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:25.063009 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:25.066583 (+ 3574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:25.066602 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:25.066621 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:25.066686 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:25.070493 (+ 3807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:25.071444 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:25.071454 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:25.071457 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:25.071617 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:25.072027 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:25.072032 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:25.072036 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:25.072494 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:25.072795 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:25.073282 (+ 487us) server_negotiation.cc:299] Negotiation successful
0619 02:53:25.073502 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 02:53:25.074739 32192 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:53:25.222522 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 32057
I20260619 02:53:25.242419 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 32201
I20260619 02:53:25.257442 32350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:25.243283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35674 (local address 127.18.134.3:42699)
0619 02:53:25.243497 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:25.243503 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:25.243526 (+ 23us) server_negotiation.cc:407] Connection header received
0619 02:53:25.243598 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:25.243604 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:25.243679 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:25.243779 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:25.244962 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:25.246057 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:25.247457 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:25.248132 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:25.248266 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:25.250681 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:25.250699 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:25.250718 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:25.250759 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:25.254540 (+ 3781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:25.255321 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:25.255330 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:25.255334 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:25.255503 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:25.255953 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:25.255958 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:25.255962 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:25.256412 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:25.256714 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:25.256879 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 02:53:25.257094 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 02:53:25.258083 32326 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:53:25.374279 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 32201
I20260619 02:53:25.390185 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 31823
I20260619 02:53:25.391415 31886 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:53:25.499898 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 31823
I20260619 02:53:25.520866 18968 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 02:53:25.856269 18968 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 02:53:25.856519 18968 mini_ranger.cc:66] Stopping Ranger...
I20260619 02:53:26.191843 18968 mini_ranger.cc:68] Stopped Ranger
2026-06-19 02:53:26.192 UTC [30875] LOG: received smart shutdown request
2026-06-19 02:53:26.195 UTC [30875] LOG: background worker "logical replication launcher" (PID 30884) exited with exit code 1
2026-06-19 02:53:26.195 UTC [30879] LOG: shutting down
2026-06-19 02:53:26.360 UTC [30875] LOG: database system is shut down
2026-06-19T02:53:26Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (53072 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:53:26 dist-test-slave-9mr0 krb5kdc[32400](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:53:26 dist-test-slave-9mr0 krb5kdc[32400](info): set up 2 sockets
Jun 19 02:53:26 dist-test-slave-9mr0 krb5kdc[32400](info): commencing operation
krb5kdc: starting...
W20260619 02:53:27.478811 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.979s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:53:27 dist-test-slave-9mr0 krb5kdc[32400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:53:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:53:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:53:27.688639 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:32875
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:45075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:32875,127.18.134.61:43329,127.18.134.60:44773
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:53:28.001950 32413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:28.002243 32413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:28.002381 32413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:53:28.010545 32413 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:53:28.010704 32413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:28.010797 32413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:28.010880 32413 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:53:28.010946 32413 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:53:28.021211 32413 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:45075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:32875,127.18.134.61:43329,127.18.134.60:44773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:32875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:28.022589 32413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:28.023945 32413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:53:28.035782 32418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:28.035830 32421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:28.035755 32419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:28.036898 32413 server_base.cc:1034] running on GCE node
I20260619 02:53:28.037652 32413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:28.039081 32413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:28.040289 32413 hybrid_clock.cc:648] HybridClock initialized: now 1781837608040230 us; error 48 us; skew 500 ppm
I20260619 02:53:28.040634 32413 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:28 dist-test-slave-9mr0 krb5kdc[32400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837608, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:28.043608 32413 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:53:28.045040 32413 webserver.cc:460] Webserver started at http://127.18.134.62:39787/ using document root <none> and password file <none>
I20260619 02:53:28.045683 32413 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:28.045789 32413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:53:28.046074 32413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:53:28.049000 32413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "47ee2de45aa9473db93d0a81b527317a"
format_stamp: "Formatted at 2026-06-19 02:53:28 on dist-test-slave-9mr0"
server_key: "65ace4d3eebbb703191409acfb662dba"
server_key_iv: "89223d83fdf08907d5ae57104527f058"
server_key_version: "encryptionkey@0"
I20260619 02:53:28.049765 32413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "47ee2de45aa9473db93d0a81b527317a"
format_stamp: "Formatted at 2026-06-19 02:53:28 on dist-test-slave-9mr0"
server_key: "65ace4d3eebbb703191409acfb662dba"
server_key_iv: "89223d83fdf08907d5ae57104527f058"
server_key_version: "encryptionkey@0"
I20260619 02:53:28.054525 32413 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260619 02:53:28.057804 32428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:28.059154 32413 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:53:28.059312 32413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "47ee2de45aa9473db93d0a81b527317a"
format_stamp: "Formatted at 2026-06-19 02:53:28 on dist-test-slave-9mr0"
server_key: "65ace4d3eebbb703191409acfb662dba"
server_key_iv: "89223d83fdf08907d5ae57104527f058"
server_key_version: "encryptionkey@0"
I20260619 02:53:28.059484 32413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:28.077828 32413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:28.081723 32413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:28.081979 32413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:28.099314 32413 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:32875
I20260619 02:53:28.099363 32479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:32875 every 8 connection(s)
I20260619 02:53:28.100641 32413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:53:28.104105 32480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:53:28.108659 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 32413
I20260619 02:53:28.108875 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:53:28.109644 18968 external_mini_cluster.cc:1428] Setting key 4f86cef9c4919d29333e2386d14c0790
I20260619 02:53:28.111493 32480 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32875 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:53:28 dist-test-slave-9mr0 krb5kdc[32400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837608, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.134.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.61@KRBTEST.COM" created.
I20260619 02:53:28.134569 32483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.115151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45726 (local address 127.18.134.62:32875)
0619 02:53:28.116972 (+ 1821us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:28.116980 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:28.117041 (+ 61us) server_negotiation.cc:407] Connection header received
0619 02:53:28.118422 (+ 1381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:28.118429 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:28.118758 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:28.118944 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:28.120017 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:28.121719 (+ 1702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:28.123096 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:28.123877 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:28.124039 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:28.127341 (+ 3302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:28.127351 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:28.127367 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:28.127407 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:28.130572 (+ 3165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:28.131263 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:28.131271 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:28.131274 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:28.131436 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:28.131823 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:28.131827 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:28.131829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:28.132438 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:28.132639 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:28.133247 (+ 608us) server_negotiation.cc:299] Negotiation successful
0619 02:53:28.133498 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1615,"spinlock_wait_cycles":37248,"thread_start_us":1378,"threads_started":1}
I20260619 02:53:28.134594 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.115151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:32875 (local address 127.0.0.1:45726)
0619 02:53:28.116047 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.116093 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:28.118068 (+ 1975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:28.118963 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:28.118968 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:28.119505 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:53:28.119835 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:28.119852 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:28.121866 (+ 2014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:28.121870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:28.122974 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:28.122986 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:28.124118 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:28.124123 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:28.124270 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:28.124884 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:53:28.124906 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:53:28.127107 (+ 2201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:53:28.130725 (+ 3618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:28.130735 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:28.130744 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:28.131113 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:28.131551 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:28.131555 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:28.131557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:28.131712 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:28.132590 (+ 878us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:53:28.132598 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:53:28.132861 (+ 263us) client_negotiation.cc:769] Sending connection context
0619 02:53:28.133128 (+ 267us) client_negotiation.cc:240] Negotiation successful
0619 02:53:28.133692 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"mutex_wait_us":70,"thread_start_us":390,"threads_started":1}
I20260619 02:53:28.137280 32480 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:53:28.138298 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.137862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:43329 (local address 127.0.0.1:39322)
0619 02:53:28.138047 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.138169 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:43329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260619 02:53:28.138532 32431 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.134.61:43329: connect: Connection refused (error 111)
I20260619 02:53:28.140025 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.139666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:43329 (local address 127.0.0.1:39328)
0619 02:53:28.139835 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.139905 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:43329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260619 02:53:28.140237 32480 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.61:43329: Network error: Client connection negotiation failed: client connection to 127.18.134.61:43329: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.61.keytab.
Entry for principal kudu/127.18.134.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.61.keytab.
I20260619 02:53:28.158033 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.61:43329
--webserver_interface=127.18.134.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.61.keytab
--principal=kudu/127.18.134.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:45075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:32875,127.18.134.61:43329,127.18.134.60:44773
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:53:28.189551 32480 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 } attempt: 1
I20260619 02:53:28.190800 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.190266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:43329 (local address 127.0.0.1:39332)
0619 02:53:28.190462 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.190589 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:43329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260619 02:53:28.191706 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.191383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:43329 (local address 127.0.0.1:39334)
0619 02:53:28.191522 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.191587 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:43329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260619 02:53:28.191892 32480 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.61:43329: Network error: Client connection negotiation failed: client connection to 127.18.134.61:43329: connect: Connection refused (error 111)
I20260619 02:53:28.260154 32480 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 } attempt: 2
I20260619 02:53:28.261379 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.260887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:43329 (local address 127.0.0.1:39340)
0619 02:53:28.261143 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.261234 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:43329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260619 02:53:28.262374 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.261998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:43329 (local address 127.0.0.1:39342)
0619 02:53:28.262146 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.262229 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:43329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260619 02:53:28.262596 32480 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.61:43329: Network error: Client connection negotiation failed: client connection to 127.18.134.61:43329: connect: Connection refused (error 111)
I20260619 02:53:28.353891 32480 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 } attempt: 3
I20260619 02:53:28.355152 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.354675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:43329 (local address 127.0.0.1:39352)
0619 02:53:28.354912 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.355021 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:43329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260619 02:53:28.356194 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.355845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:43329 (local address 127.0.0.1:39362)
0619 02:53:28.355989 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.356058 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:43329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260619 02:53:28.356383 32480 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.61:43329: Network error: Client connection negotiation failed: client connection to 127.18.134.61:43329: connect: Connection refused (error 111)
W20260619 02:53:28.471138 32487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:28.471473 32487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:28.471609 32487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:53:28.478982 32487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:53:28.479110 32487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:28.479188 32487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:28.479270 32487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:53:28.479333 32487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:53:28.489285 32487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:45075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:32875,127.18.134.61:43329,127.18.134.60:44773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.61.keytab
--principal=kudu/127.18.134.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.61:43329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.134.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:28.490618 32487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:28.491894 32487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:53:28.499609 32480 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 } attempt: 4
I20260619 02:53:28.500715 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.500207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:43329 (local address 127.0.0.1:39370)
0619 02:53:28.500487 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.500570 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:43329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260619 02:53:28.501803 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.501340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:43329 (local address 127.0.0.1:39374)
0619 02:53:28.501540 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.501654 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:43329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260619 02:53:28.502143 32480 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.61:43329: Network error: Client connection negotiation failed: client connection to 127.18.134.61:43329: connect: Connection refused (error 111)
W20260619 02:53:28.503476 32492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:28.503399 32493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:28.503897 32495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:28.504262 32487 server_base.cc:1034] running on GCE node
I20260619 02:53:28.504985 32487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:28.506388 32487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:28.507591 32487 hybrid_clock.cc:648] HybridClock initialized: now 1781837608507535 us; error 48 us; skew 500 ppm
I20260619 02:53:28.507923 32487 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:28 dist-test-slave-9mr0 krb5kdc[32400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837608, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:28.510985 32487 init.cc:377] Logged in from keytab as kudu/127.18.134.61@KRBTEST.COM (short username kudu)
I20260619 02:53:28.512542 32487 webserver.cc:460] Webserver started at http://127.18.134.61:40021/ using document root <none> and password file <none>
I20260619 02:53:28.513212 32487 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:28.513331 32487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:53:28.513617 32487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:53:28.516604 32487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-1/data/instance:
uuid: "b24f3658c5a146919e1ac01f6a2e6505"
format_stamp: "Formatted at 2026-06-19 02:53:28 on dist-test-slave-9mr0"
server_key: "9687676e31532fd294b452263f912781"
server_key_iv: "116f475285ad5e70b0743a2de55b37ec"
server_key_version: "encryptionkey@0"
I20260619 02:53:28.517374 32487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-1/wal/instance:
uuid: "b24f3658c5a146919e1ac01f6a2e6505"
format_stamp: "Formatted at 2026-06-19 02:53:28 on dist-test-slave-9mr0"
server_key: "9687676e31532fd294b452263f912781"
server_key_iv: "116f475285ad5e70b0743a2de55b37ec"
server_key_version: "encryptionkey@0"
I20260619 02:53:28.522398 32487 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 02:53:28.525741 32502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:28.527096 32487 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:53:28.527254 32487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-1/wal
uuid: "b24f3658c5a146919e1ac01f6a2e6505"
format_stamp: "Formatted at 2026-06-19 02:53:28 on dist-test-slave-9mr0"
server_key: "9687676e31532fd294b452263f912781"
server_key_iv: "116f475285ad5e70b0743a2de55b37ec"
server_key_version: "encryptionkey@0"
I20260619 02:53:28.527434 32487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:28.552030 32487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:28.556073 32487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:28.556348 32487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:28.573673 32487 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.61:43329
I20260619 02:53:28.573730 32553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.61:43329 every 8 connection(s)
I20260619 02:53:28.575018 32487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-1/data/info.pb
I20260619 02:53:28.577443 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 32487
I20260619 02:53:28.577592 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-1/wal/instance
I20260619 02:53:28.578025 18968 external_mini_cluster.cc:1428] Setting key bcad4d441b7905f8be9e780c15bb0dab
I20260619 02:53:28.578747 32554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:53:28.584908 32554 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32875 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.134.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.60@KRBTEST.COM" created.
Jun 19 02:53:28 dist-test-slave-9mr0 krb5kdc[32400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837608, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.61@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:53:28.605325 32483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.587501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45730 (local address 127.18.134.62:32875)
0619 02:53:28.587666 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:28.587670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:28.588950 (+ 1280us) server_negotiation.cc:407] Connection header received
0619 02:53:28.590766 (+ 1816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:28.590772 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:28.590853 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:28.590941 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:28.592527 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:28.593259 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:28.594875 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:28.595359 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:28.595466 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:28.598916 (+ 3450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:28.598922 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:28.598926 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:28.598968 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:28.602096 (+ 3128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:28.602915 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:28.602921 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:28.602924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:28.603065 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:28.603487 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:28.603491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:28.603494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:28.603842 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:28.603984 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:28.604776 (+ 792us) server_negotiation.cc:299] Negotiation successful
0619 02:53:28.605017 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 02:53:28.606268 32557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.588083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:32875 (local address 127.0.0.1:45730)
0619 02:53:28.588790 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.588826 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:28.590490 (+ 1664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:28.591174 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:28.591182 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:28.591689 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:53:28.592259 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:28.592276 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:28.593418 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:28.593422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:28.594712 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:28.594724 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:28.595562 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:28.595567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:28.595758 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:28.596409 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:53:28.596436 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:53:28.598676 (+ 2240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:53:28.602253 (+ 3577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:28.602261 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:28.602275 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:28.602792 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:28.603172 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:28.603175 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:28.603177 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:28.603384 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:28.603956 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:53:28.603961 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:53:28.604389 (+ 428us) client_negotiation.cc:769] Sending connection context
0619 02:53:28.604635 (+ 246us) client_negotiation.cc:240] Negotiation successful
0619 02:53:28.605146 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":292,"threads_started":1}
I20260619 02:53:28.608098 32554 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:53:28 dist-test-slave-9mr0 krb5kdc[32400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837608, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.61@KRBTEST.COM for kudu/127.18.134.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.60.keytab.
Entry for principal kudu/127.18.134.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.60.keytab.
I20260619 02:53:28.623009 32557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.608795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:43329 (local address 127.0.0.1:39390)
0619 02:53:28.608955 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.608974 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:28.609070 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:28.610122 (+ 1052us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:28.610126 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:28.610581 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:53:28.610766 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:28.610775 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:28.612054 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:28.612059 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:28.612879 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:28.612889 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:28.613638 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:28.613643 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:28.613838 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:28.614377 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:53:28.614393 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:53:28.617953 (+ 3560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:53:28.620746 (+ 2793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:28.620752 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:28.620756 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:28.621079 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:28.621416 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:28.621421 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:28.621425 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:28.621539 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:28.622088 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:53:28.622092 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:53:28.622224 (+ 132us) client_negotiation.cc:769] Sending connection context
0619 02:53:28.622343 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 02:53:28.622588 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260619 02:53:28.623077 32559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.609074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39390 (local address 127.18.134.61:43329)
0619 02:53:28.609722 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:28.609728 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:28.609752 (+ 24us) server_negotiation.cc:407] Connection header received
0619 02:53:28.609791 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:28.609815 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:28.609985 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:28.610109 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:28.610902 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:28.611931 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:28.613001 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:28.613509 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:28.613614 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:28.618138 (+ 4524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:28.618149 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:28.618162 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:28.618188 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:28.620631 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:28.621193 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:28.621199 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:28.621201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:28.621308 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:28.621638 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:28.621643 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:28.621647 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:28.621975 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:28.622160 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:28.622491 (+ 331us) server_negotiation.cc:299] Negotiation successful
0619 02:53:28.622715 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":358,"threads_started":1}
I20260619 02:53:28.623833 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.60:44773
--webserver_interface=127.18.134.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.60.keytab
--principal=kudu/127.18.134.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:45075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:32875,127.18.134.61:43329,127.18.134.60:44773
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:53:28.625564 32554 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:53:28.626807 32557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.626310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44773 (local address 127.0.0.1:40262)
0619 02:53:28.626476 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.626612 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260619 02:53:28.627074 32503 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44773: connect: Connection refused (error 111)
I20260619 02:53:28.628523 32557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.628195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44773 (local address 127.0.0.1:40266)
0619 02:53:28.628351 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.628422 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260619 02:53:28.628728 32554 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.60:44773: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44773: connect: Connection refused (error 111)
I20260619 02:53:28.678042 32554 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } attempt: 1
I20260619 02:53:28.679344 32557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.678814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44773 (local address 127.0.0.1:40278)
0619 02:53:28.679025 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.679147 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260619 02:53:28.680302 32557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.679915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44773 (local address 127.0.0.1:40294)
0619 02:53:28.680079 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.680160 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260619 02:53:28.680500 32554 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.60:44773: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44773: connect: Connection refused (error 111)
I20260619 02:53:28.748739 32554 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } attempt: 2
I20260619 02:53:28.749938 32557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.749424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44773 (local address 127.0.0.1:40300)
0619 02:53:28.749638 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.749751 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260619 02:53:28.750949 32557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.750520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44773 (local address 127.0.0.1:40312)
0619 02:53:28.750721 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.750806 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260619 02:53:28.751170 32554 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.60:44773: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44773: connect: Connection refused (error 111)
I20260619 02:53:28.801412 32480 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 } attempt: 5
Jun 19 02:53:28 dist-test-slave-9mr0 krb5kdc[32400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837608, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for kudu/127.18.134.61@KRBTEST.COM
I20260619 02:53:28.815616 32559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.802352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39398 (local address 127.18.134.61:43329)
0619 02:53:28.802506 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:28.802511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:28.802530 (+ 19us) server_negotiation.cc:407] Connection header received
0619 02:53:28.802582 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:28.802585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:28.802683 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:28.802776 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:28.803871 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:28.804585 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:28.806305 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:28.806817 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:28.806928 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:28.810524 (+ 3596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:28.810528 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:28.810531 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:28.810564 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:28.813097 (+ 2533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:28.813890 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:28.813893 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:28.813896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:28.814007 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:28.814391 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:28.814394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:28.814395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:28.814696 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:28.814801 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:28.815226 (+ 425us) server_negotiation.cc:299] Negotiation successful
0619 02:53:28.815390 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 02:53:28.815810 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.802037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:43329 (local address 127.0.0.1:39398)
0619 02:53:28.802289 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.802314 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:28.802458 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:28.802852 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:28.802857 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:28.803388 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:53:28.803651 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:28.803665 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:28.804747 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:28.804754 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:28.806150 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:28.806166 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:28.806976 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:28.806983 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:28.807187 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:28.807852 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:53:28.807874 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:53:28.810319 (+ 2445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:53:28.813234 (+ 2915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:28.813239 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:28.813244 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:28.813786 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:28.814119 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:28.814124 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:28.814128 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:28.814293 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:28.814825 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:53:28.814830 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:53:28.815007 (+ 177us) client_negotiation.cc:769] Sending connection context
0619 02:53:28.815159 (+ 152us) client_negotiation.cc:240] Negotiation successful
0619 02:53:28.815451 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260619 02:53:28.816864 32480 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:53:28.817941 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.817571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44773 (local address 127.0.0.1:40324)
0619 02:53:28.817721 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.817794 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260619 02:53:28.818862 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.818510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44773 (local address 127.0.0.1:40332)
0619 02:53:28.818678 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.818752 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260619 02:53:28.819041 32480 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.60:44773: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44773: connect: Connection refused (error 111)
I20260619 02:53:28.842407 32554 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } attempt: 3
I20260619 02:53:28.843621 32557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.843063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44773 (local address 127.0.0.1:40338)
0619 02:53:28.843289 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.843402 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260619 02:53:28.844707 32557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.844256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44773 (local address 127.0.0.1:40350)
0619 02:53:28.844477 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.844566 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
W20260619 02:53:28.844920 32554 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.60:44773: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44773: connect: Connection refused (error 111)
I20260619 02:53:28.870260 32480 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } attempt: 1
I20260619 02:53:28.871318 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.870873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44773 (local address 127.0.0.1:40364)
0619 02:53:28.871056 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.871162 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260619 02:53:28.872370 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.872005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44773 (local address 127.0.0.1:40368)
0619 02:53:28.872168 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.872240 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260619 02:53:28.872577 32480 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.60:44773: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44773: connect: Connection refused (error 111)
W20260619 02:53:28.935372 32561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:28.935636 32561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:28.935768 32561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260619 02:53:28.940783 32480 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } attempt: 2
I20260619 02:53:28.941884 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.941452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44773 (local address 127.0.0.1:40372)
0619 02:53:28.941630 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.941730 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260619 02:53:28.942806 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.942452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44773 (local address 127.0.0.1:40376)
0619 02:53:28.942592 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.942699 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260619 02:53:28.943048 32480 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.60:44773: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44773: connect: Connection refused (error 111)
W20260619 02:53:28.943117 32561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:53:28.943241 32561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:28.943312 32561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:28.943387 32561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:53:28.943446 32561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:53:28.953214 32561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:45075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:32875,127.18.134.61:43329,127.18.134.60:44773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.60.keytab
--principal=kudu/127.18.134.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.60:44773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.134.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:28.954552 32561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:28.955847 32561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:53:28.967132 32566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:28.967115 32569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:28.967134 32567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:28.968865 32561 server_base.cc:1034] running on GCE node
I20260619 02:53:28.969653 32561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:28.971061 32561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:28.972291 32561 hybrid_clock.cc:648] HybridClock initialized: now 1781837608972222 us; error 62 us; skew 500 ppm
I20260619 02:53:28.972618 32561 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:28 dist-test-slave-9mr0 krb5kdc[32400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837608, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:28.975622 32561 init.cc:377] Logged in from keytab as kudu/127.18.134.60@KRBTEST.COM (short username kudu)
I20260619 02:53:28.977036 32561 webserver.cc:460] Webserver started at http://127.18.134.60:33711/ using document root <none> and password file <none>
I20260619 02:53:28.977700 32561 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:28.977815 32561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:53:28.978098 32561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:53:28.980979 32561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-2/data/instance:
uuid: "635815b906114ae1b5dd994f0a71a4e4"
format_stamp: "Formatted at 2026-06-19 02:53:28 on dist-test-slave-9mr0"
server_key: "7f0427356c4de8e9b5ae7fdbbf16565a"
server_key_iv: "add5d93185f67f69c127980fc5a8506c"
server_key_version: "encryptionkey@0"
I20260619 02:53:28.981756 32561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-2/wal/instance:
uuid: "635815b906114ae1b5dd994f0a71a4e4"
format_stamp: "Formatted at 2026-06-19 02:53:28 on dist-test-slave-9mr0"
server_key: "7f0427356c4de8e9b5ae7fdbbf16565a"
server_key_iv: "add5d93185f67f69c127980fc5a8506c"
server_key_version: "encryptionkey@0"
I20260619 02:53:28.986615 32561 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 02:53:28.988109 32554 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } attempt: 4
I20260619 02:53:28.989210 32557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.988714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44773 (local address 127.0.0.1:40390)
0619 02:53:28.988921 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.989027 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260619 02:53:28.990131 32576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:28.990149 32557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:28.989859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44773 (local address 127.0.0.1:40394)
0619 02:53:28.989984 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:28.990050 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 02:53:28.990367 32554 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.60:44773: Network error: Client connection negotiation failed: client connection to 127.18.134.60:44773: connect: Connection refused (error 111)
I20260619 02:53:28.991479 32561 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260619 02:53:28.991612 32561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-2/wal
uuid: "635815b906114ae1b5dd994f0a71a4e4"
format_stamp: "Formatted at 2026-06-19 02:53:28 on dist-test-slave-9mr0"
server_key: "7f0427356c4de8e9b5ae7fdbbf16565a"
server_key_iv: "add5d93185f67f69c127980fc5a8506c"
server_key_version: "encryptionkey@0"
I20260619 02:53:28.991811 32561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:29.011987 32561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:29.016247 32561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:29.016496 32561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:29.032970 32561 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.60:44773
I20260619 02:53:29.032992 32627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.60:44773 every 8 connection(s)
I20260619 02:53:29.034250 32561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-2/data/info.pb
I20260619 02:53:29.037858 32628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:53:29.042804 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 32561
I20260619 02:53:29.042985 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781837410515220-18968-0/minicluster-data/master-2/wal/instance
I20260619 02:53:29.043417 18968 external_mini_cluster.cc:1428] Setting key 552e0d1f4667c2c39f8455f1953c7c70
I20260619 02:53:29.044431 32628 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32875 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:53:29.049274 32480 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } attempt: 3
Jun 19 02:53:29 dist-test-slave-9mr0 krb5kdc[32400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837608, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for kudu/127.18.134.60@KRBTEST.COM
Jun 19 02:53:29 dist-test-slave-9mr0 krb5kdc[32400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.60@KRBTEST.COM
Jun 19 02:53:29 dist-test-slave-9mr0 krb5kdc[32400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837608, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.60@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:53:29.067402 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:29.049751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44773 (local address 127.0.0.1:40406)
0619 02:53:29.050004 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:29.050023 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:29.050121 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:29.050907 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:29.050912 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:29.051304 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:53:29.051468 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:29.051476 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.052792 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.052798 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:29.054242 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:29.054257 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.055219 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.055225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:29.055366 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:29.055859 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:53:29.055880 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:53:29.059428 (+ 3548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:53:29.063696 (+ 4268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:29.063701 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:29.063705 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:29.064470 (+ 765us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:29.065011 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:29.065017 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:29.065021 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:29.065184 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:29.066234 (+ 1050us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:53:29.066240 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:53:29.066427 (+ 187us) client_negotiation.cc:769] Sending connection context
0619 02:53:29.066699 (+ 272us) client_negotiation.cc:240] Negotiation successful
0619 02:53:29.066983 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":25}
I20260619 02:53:29.068692 32483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:29.047251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45736 (local address 127.18.134.62:32875)
0619 02:53:29.047458 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:29.047464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:29.048185 (+ 721us) server_negotiation.cc:407] Connection header received
0619 02:53:29.048594 (+ 409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:29.048599 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:29.048674 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:29.048781 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:29.050446 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.051555 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.053493 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.054091 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.054378 (+ 287us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:29.060631 (+ 6253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:29.060638 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:29.060642 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:29.060678 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:29.065701 (+ 5023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:29.066531 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:29.066536 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:29.066540 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:29.066723 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:29.067154 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:29.067159 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:29.067162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:29.067476 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:29.067599 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:29.068114 (+ 515us) server_negotiation.cc:299] Negotiation successful
0619 02:53:29.068310 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":160,"server-negotiator.queue_time_us":58}
I20260619 02:53:29.068958 32633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:29.049769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40406 (local address 127.18.134.60:44773)
0619 02:53:29.050649 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:29.050654 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:29.050674 (+ 20us) server_negotiation.cc:407] Connection header received
0619 02:53:29.050725 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:29.050728 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:29.050773 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:29.050854 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:29.051617 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.052652 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.054421 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.055067 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.055177 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:29.059711 (+ 4534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:29.059722 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:29.059738 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:29.059778 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:29.063544 (+ 3766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:29.064641 (+ 1097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:29.064649 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:29.064658 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:29.064872 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:29.065335 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:29.065340 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:29.065343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:29.066070 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:29.066282 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:29.066945 (+ 663us) server_negotiation.cc:299] Negotiation successful
0619 02:53:29.067522 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":763,"thread_start_us":592,"threads_started":1}
I20260619 02:53:29.068976 32632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:29.047262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:32875 (local address 127.0.0.1:45736)
0619 02:53:29.048040 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:29.048087 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:29.048386 (+ 299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:29.048866 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:29.048873 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:29.049663 (+ 790us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:53:29.050239 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:29.050264 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.051771 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.051777 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:29.053346 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:29.053361 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.054482 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.054492 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:29.054695 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:29.055735 (+ 1040us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:53:29.057210 (+ 1475us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:53:29.060391 (+ 3181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:53:29.065825 (+ 5434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:29.065835 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:29.065842 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:29.066393 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:29.066858 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:29.066862 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:29.066865 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:29.067026 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:29.067700 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:53:29.067707 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:53:29.067863 (+ 156us) client_negotiation.cc:769] Sending connection context
0619 02:53:29.068017 (+ 154us) client_negotiation.cc:240] Negotiation successful
0619 02:53:29.068327 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":23,"thread_start_us":392,"threads_started":1}
I20260619 02:53:29.069185 32631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:29.045239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40404 (local address 127.18.134.60:44773)
0619 02:53:29.046060 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:29.046074 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:29.046121 (+ 47us) server_negotiation.cc:407] Connection header received
0619 02:53:29.048203 (+ 2082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:29.048238 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:29.048476 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:29.048665 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:29.050351 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.052099 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.053666 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.054504 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.054666 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:29.060723 (+ 6057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:29.060736 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:29.060740 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:29.060782 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:29.064666 (+ 3884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:29.065405 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:29.065410 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:29.065414 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:29.065576 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:29.066003 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:29.066008 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:29.066011 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:29.066355 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:29.066567 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:29.067833 (+ 1266us) server_negotiation.cc:299] Negotiation successful
0619 02:53:29.068027 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":594,"spinlock_wait_cycles":18688,"thread_start_us":350,"threads_started":1}
I20260619 02:53:29.070871 32628 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:53:29.076473 32480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47ee2de45aa9473db93d0a81b527317a: Bootstrap starting.
Jun 19 02:53:29 dist-test-slave-9mr0 krb5kdc[32400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837608, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.60@KRBTEST.COM for kudu/127.18.134.61@KRBTEST.COM
I20260619 02:53:29.080084 32480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47ee2de45aa9473db93d0a81b527317a: Neither blocks nor log segments found. Creating new log.
Jun 19 02:53:29 dist-test-slave-9mr0 krb5kdc[32400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:53:29.081337 32480 log.cc:824] T 00000000000000000000000000000000 P 47ee2de45aa9473db93d0a81b527317a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:53:29.085325 32632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:29.071682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:43329 (local address 127.0.0.1:39412)
0619 02:53:29.071854 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:29.071880 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:29.071985 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:29.072460 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:29.072465 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:29.072878 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:53:29.073061 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:29.073072 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.073981 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.073987 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:29.075151 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:29.075165 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.076060 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.076066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:29.076255 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:29.076875 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:53:29.076897 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:53:29.079125 (+ 2228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:53:29.082915 (+ 3790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:29.082921 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:29.082925 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:29.083315 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:29.083727 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:29.083732 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:29.083736 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:29.083877 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:29.084438 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:53:29.084443 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:53:29.084575 (+ 132us) client_negotiation.cc:769] Sending connection context
0619 02:53:29.084695 (+ 120us) client_negotiation.cc:240] Negotiation successful
0619 02:53:29.084961 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 02:53:29.085356 32559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:29.071620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39412 (local address 127.18.134.61:43329)
0619 02:53:29.071805 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:29.071810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:29.071944 (+ 134us) server_negotiation.cc:407] Connection header received
0619 02:53:29.072106 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:29.072112 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:29.072179 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:29.072266 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:29.073178 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.073857 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.075298 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.075905 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.076035 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:29.079323 (+ 3288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:29.079330 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:29.079334 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:29.079375 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:29.082750 (+ 3375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:29.083433 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:29.083438 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:29.083441 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:29.083613 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:29.083981 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:29.083986 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:29.083990 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:29.084249 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:29.084387 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:29.084806 (+ 419us) server_negotiation.cc:299] Negotiation successful
0619 02:53:29.085003 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 02:53:29.085970 32480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47ee2de45aa9473db93d0a81b527317a: No bootstrap required, opened a new log
I20260619 02:53:29.086313 32628 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:53:29.088160 32483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:29.072360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45750 (local address 127.18.134.62:32875)
0619 02:53:29.072515 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:29.072521 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:29.072634 (+ 113us) server_negotiation.cc:407] Connection header received
0619 02:53:29.072778 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:29.072784 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:29.072871 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:29.072972 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:29.074441 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.075485 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.076985 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.077594 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.077714 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:29.081936 (+ 4222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:29.081941 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:29.081944 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:29.081973 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:29.085121 (+ 3148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:29.085929 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:29.085933 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:29.085936 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:29.086114 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:29.086596 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:29.086600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:29.086603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:29.087022 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:29.087290 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:29.087611 (+ 321us) server_negotiation.cc:299] Negotiation successful
0619 02:53:29.087828 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Jun 19 02:53:29 dist-test-slave-9mr0 krb5kdc[32400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837608, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.60@KRBTEST.COM for kudu/127.18.134.60@KRBTEST.COM
I20260619 02:53:29.094844 32480 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47ee2de45aa9473db93d0a81b527317a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47ee2de45aa9473db93d0a81b527317a" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32875 } } peers { permanent_uuid: "b24f3658c5a146919e1ac01f6a2e6505" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 } } peers { permanent_uuid: "635815b906114ae1b5dd994f0a71a4e4" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } }
I20260619 02:53:29.095146 32480 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47ee2de45aa9473db93d0a81b527317a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:53:29.095242 32480 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47ee2de45aa9473db93d0a81b527317a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47ee2de45aa9473db93d0a81b527317a, State: Initialized, Role: FOLLOWER
I20260619 02:53:29.095811 32480 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47ee2de45aa9473db93d0a81b527317a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47ee2de45aa9473db93d0a81b527317a" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32875 } } peers { permanent_uuid: "b24f3658c5a146919e1ac01f6a2e6505" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 } } peers { permanent_uuid: "635815b906114ae1b5dd994f0a71a4e4" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } }
Jun 19 02:53:29 dist-test-slave-9mr0 krb5kdc[32400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.61@KRBTEST.COM
I20260619 02:53:29.100088 32480 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47ee2de45aa9473db93d0a81b527317a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:53:29.102555 32632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:29.086989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44773 (local address 127.0.0.1:40416)
0619 02:53:29.087331 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:29.087348 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:29.087453 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:29.087838 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:29.087843 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:29.088256 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:53:29.088427 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:29.088438 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.089341 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.089347 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:29.091080 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:29.091095 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.092055 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.092061 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:29.092283 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:29.092850 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:53:29.092871 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:53:29.095721 (+ 2850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:53:29.099781 (+ 4060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:29.099785 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:29.099789 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:29.100330 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:29.100728 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:29.100733 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:29.100736 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:29.100889 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:29.101556 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:53:29.101561 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:53:29.101711 (+ 150us) client_negotiation.cc:769] Sending connection context
0619 02:53:29.101857 (+ 146us) client_negotiation.cc:240] Negotiation successful
0619 02:53:29.102188 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266}
I20260619 02:53:29.102769 32633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:29.087000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40416 (local address 127.18.134.60:44773)
0619 02:53:29.087235 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:29.087240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:29.087423 (+ 183us) server_negotiation.cc:407] Connection header received
0619 02:53:29.087568 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:29.087572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:29.087643 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:29.087711 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:29.088552 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.089230 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.091238 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.091905 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.092030 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:29.096185 (+ 4155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:29.096191 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:29.096196 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:29.096256 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:29.099647 (+ 3391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:29.100455 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:29.100460 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:29.100463 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:29.100610 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:29.101003 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:29.101009 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:29.101012 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:29.101427 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:29.101565 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:29.102014 (+ 449us) server_negotiation.cc:299] Negotiation successful
0619 02:53:29.102383 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":91,"server-negotiator.queue_time_us":126}
I20260619 02:53:29.106925 32635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47ee2de45aa9473db93d0a81b527317a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47ee2de45aa9473db93d0a81b527317a" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32875 } } peers { permanent_uuid: "b24f3658c5a146919e1ac01f6a2e6505" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 } } peers { permanent_uuid: "635815b906114ae1b5dd994f0a71a4e4" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } } }
I20260619 02:53:29.107368 32635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47ee2de45aa9473db93d0a81b527317a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:53:29.108628 32628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4: Bootstrap starting.
I20260619 02:53:29.112648 32628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4: Neither blocks nor log segments found. Creating new log.
I20260619 02:53:29.114004 32628 log.cc:824] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4: Log is configured to *not* fsync() on all Append() calls
I20260619 02:53:29.115020 32559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:29.089948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39424 (local address 127.18.134.61:43329)
0619 02:53:29.090095 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:29.090100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:29.090126 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:53:29.090172 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:29.090176 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:29.090239 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:29.090312 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:29.091742 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.092642 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.094418 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.095096 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.095212 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:29.107853 (+ 12641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:29.107860 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:29.107864 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:29.107906 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:29.111238 (+ 3332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:29.112215 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:29.112219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:29.112222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:29.112386 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:29.112862 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:29.112867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:29.112870 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:29.113225 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:29.113478 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:29.114433 (+ 955us) server_negotiation.cc:299] Negotiation successful
0619 02:53:29.114708 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 02:53:29.119196 32628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4: No bootstrap required, opened a new log
I20260619 02:53:29.120206 18968 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:53:29.129935 32628 raft_consensus.cc:348] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47ee2de45aa9473db93d0a81b527317a" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32875 } } peers { permanent_uuid: "b24f3658c5a146919e1ac01f6a2e6505" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 } } peers { permanent_uuid: "635815b906114ae1b5dd994f0a71a4e4" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } }
I20260619 02:53:29.130491 32628 raft_consensus.cc:374] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:53:29.122728 32413 master_runner.cc:186] Error getting master registration for 127.18.134.61:43329: OK, instance_id { permanent_uuid: "b24f3658c5a146919e1ac01f6a2e6505" instance_seqno: 1781837608568740 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:53:29.130781 32413 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 02:53:29.130754 32628 raft_consensus.cc:729] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 635815b906114ae1b5dd994f0a71a4e4, State: Initialized, Role: FOLLOWER
I20260619 02:53:29.131549 32628 consensus_queue.cc:260] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47ee2de45aa9473db93d0a81b527317a" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32875 } } peers { permanent_uuid: "b24f3658c5a146919e1ac01f6a2e6505" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 } } peers { permanent_uuid: "635815b906114ae1b5dd994f0a71a4e4" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } }
I20260619 02:53:29.132534 32650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47ee2de45aa9473db93d0a81b527317a" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32875 } } peers { permanent_uuid: "b24f3658c5a146919e1ac01f6a2e6505" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 } } peers { permanent_uuid: "635815b906114ae1b5dd994f0a71a4e4" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } } }
I20260619 02:53:29.132997 32650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:53:29.133823 32628 sys_catalog.cc:564] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [sys.catalog]: configured and running, proceeding with master startup.
W20260619 02:53:29.137040 32647 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 47ee2de45aa9473db93d0a81b527317a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:53:29.137173 32647 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:53:29.156169 32559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:29.136724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39428 (local address 127.18.134.61:43329)
0619 02:53:29.136867 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:29.136871 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:29.137636 (+ 765us) server_negotiation.cc:407] Connection header received
0619 02:53:29.137956 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:29.137963 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:29.138044 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:29.138116 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:29.140865 (+ 2749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.142041 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.144201 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.144864 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.144963 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:29.148890 (+ 3927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:29.148897 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:29.148903 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:29.148948 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:29.152620 (+ 3672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:29.153626 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:29.153633 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:29.153637 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:29.153821 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:29.154263 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:29.154269 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:29.154273 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:29.154655 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:29.154924 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:29.155511 (+ 587us) server_negotiation.cc:299] Negotiation successful
0619 02:53:29.155785 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 02:53:29.165830 32483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:29.136581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45764 (local address 127.18.134.62:32875)
0619 02:53:29.136759 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:29.136765 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:29.147263 (+ 10498us) server_negotiation.cc:407] Connection header received
0619 02:53:29.147538 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:29.147545 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:29.147630 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:29.147702 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:29.149771 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.150957 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.153574 (+ 2617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.154315 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.154418 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:29.158303 (+ 3885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:29.158311 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:29.158316 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:29.158368 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:29.162130 (+ 3762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:29.163164 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:29.163170 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:29.163174 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:29.163348 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:29.163803 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:29.163808 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:29.163812 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:29.164260 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:29.164534 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:29.165138 (+ 604us) server_negotiation.cc:299] Negotiation successful
0619 02:53:29.165405 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 02:53:29.173938 32633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:29.143558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40424 (local address 127.18.134.60:44773)
0619 02:53:29.143717 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:29.143723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:29.157293 (+ 13570us) server_negotiation.cc:407] Connection header received
0619 02:53:29.157566 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:29.157572 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:29.157652 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:29.157736 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:29.159348 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.160356 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.163098 (+ 2742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.163818 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.163918 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:29.167356 (+ 3438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:29.167362 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:29.167367 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:29.167408 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:29.170953 (+ 3545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:29.171822 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:29.171828 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:29.171831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:29.171989 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:29.172404 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:29.172409 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:29.172413 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:29.172768 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:29.173031 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:29.173342 (+ 311us) server_negotiation.cc:299] Negotiation successful
0619 02:53:29.173561 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260619 02:53:29.189466 32667 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:53:29.189718 32667 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:53:29.192334 32561 master_runner.cc:186] Error getting master registration for 127.18.134.61:43329: OK, instance_id { permanent_uuid: "b24f3658c5a146919e1ac01f6a2e6505" instance_seqno: 1781837608568740 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:53:29.202852 32561 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260619 02:53:29.268651 32582 tablet.cc:2369] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:53:29.276089 32650 raft_consensus.cc:482] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:53:29.276424 32650 raft_consensus.cc:504] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47ee2de45aa9473db93d0a81b527317a" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32875 } } peers { permanent_uuid: "b24f3658c5a146919e1ac01f6a2e6505" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 } } peers { permanent_uuid: "635815b906114ae1b5dd994f0a71a4e4" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } }
I20260619 02:53:29.278183 32650 leader_election.cc:288] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47ee2de45aa9473db93d0a81b527317a (127.18.134.62:32875), b24f3658c5a146919e1ac01f6a2e6505 (127.18.134.61:43329)
I20260619 02:53:29.279129 32455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "635815b906114ae1b5dd994f0a71a4e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47ee2de45aa9473db93d0a81b527317a" is_pre_election: true
I20260619 02:53:29.279737 32455 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 47ee2de45aa9473db93d0a81b527317a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 635815b906114ae1b5dd994f0a71a4e4 in term 0.
I20260619 02:53:29.280349 32578 leader_election.cc:302] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47ee2de45aa9473db93d0a81b527317a, 635815b906114ae1b5dd994f0a71a4e4; no voters:
I20260619 02:53:29.280773 32650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:53:29.280930 32650 raft_consensus.cc:482] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:53:29.281041 32650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:53:29.284405 32650 raft_consensus.cc:504] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47ee2de45aa9473db93d0a81b527317a" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32875 } } peers { permanent_uuid: "b24f3658c5a146919e1ac01f6a2e6505" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 } } peers { permanent_uuid: "635815b906114ae1b5dd994f0a71a4e4" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } }
I20260619 02:53:29.285586 32455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "635815b906114ae1b5dd994f0a71a4e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47ee2de45aa9473db93d0a81b527317a"
I20260619 02:53:29.285714 32650 leader_election.cc:288] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [CANDIDATE]: Term 1 election: Requested vote from peers 47ee2de45aa9473db93d0a81b527317a (127.18.134.62:32875), b24f3658c5a146919e1ac01f6a2e6505 (127.18.134.61:43329)
I20260619 02:53:29.285836 32455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47ee2de45aa9473db93d0a81b527317a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:53:29.289546 32554 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } attempt: 5
I20260619 02:53:29.297314 32455 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 47ee2de45aa9473db93d0a81b527317a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 635815b906114ae1b5dd994f0a71a4e4 in term 1.
I20260619 02:53:29.299032 32578 leader_election.cc:302] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47ee2de45aa9473db93d0a81b527317a, 635815b906114ae1b5dd994f0a71a4e4; no voters:
Jun 19 02:53:29 dist-test-slave-9mr0 krb5kdc[32400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837608, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.61@KRBTEST.COM for kudu/127.18.134.60@KRBTEST.COM
I20260619 02:53:29.305043 32650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:53:29.309957 32557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:29.290981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:44773 (local address 127.0.0.1:40434)
0619 02:53:29.291195 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:29.291216 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:29.291324 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:29.291780 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:29.291785 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:29.292301 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:53:29.292506 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:29.292518 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.293957 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.293963 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:29.295510 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:29.295527 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.296788 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.296795 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:29.297029 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:29.297687 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:53:29.297711 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:53:29.302856 (+ 5145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:53:29.307064 (+ 4208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:29.307069 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:29.307074 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:29.307600 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:29.308043 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:29.308048 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:29.308052 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:29.308213 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:29.308948 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:53:29.308953 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:53:29.309110 (+ 157us) client_negotiation.cc:769] Sending connection context
0619 02:53:29.309257 (+ 147us) client_negotiation.cc:240] Negotiation successful
0619 02:53:29.309551 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260619 02:53:29.311023 32633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:29.291312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40434 (local address 127.18.134.60:44773)
0619 02:53:29.291477 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:29.291483 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:29.291501 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:53:29.291557 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:29.291563 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:29.291642 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:29.291744 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:29.292650 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.293826 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.295662 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:29.296385 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:29.296513 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:29.303134 (+ 6621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:29.303142 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:29.303147 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:29.303192 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:29.306896 (+ 3704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:29.307746 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:29.307752 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:29.307767 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:29.307912 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:29.308337 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:29.308342 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:29.308346 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:29.308736 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:29.308883 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:29.310385 (+ 1502us) server_negotiation.cc:299] Negotiation successful
0619 02:53:29.310625 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 02:53:29.311537 32650 raft_consensus.cc:686] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [term 1 LEADER]: Becoming Leader. State: Replica: 635815b906114ae1b5dd994f0a71a4e4, State: Running, Role: LEADER
I20260619 02:53:29.312651 32650 consensus_queue.cc:237] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47ee2de45aa9473db93d0a81b527317a" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32875 } } peers { permanent_uuid: "b24f3658c5a146919e1ac01f6a2e6505" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 } } peers { permanent_uuid: "635815b906114ae1b5dd994f0a71a4e4" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } }
I20260619 02:53:29.317518 32554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b24f3658c5a146919e1ac01f6a2e6505: Bootstrap starting.
I20260619 02:53:29.322528 32554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b24f3658c5a146919e1ac01f6a2e6505: Neither blocks nor log segments found. Creating new log.
I20260619 02:53:29.324082 32554 log.cc:824] T 00000000000000000000000000000000 P b24f3658c5a146919e1ac01f6a2e6505: Log is configured to *not* fsync() on all Append() calls
I20260619 02:53:29.326293 32669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 635815b906114ae1b5dd994f0a71a4e4. Latest consensus state: current_term: 1 leader_uuid: "635815b906114ae1b5dd994f0a71a4e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47ee2de45aa9473db93d0a81b527317a" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32875 } } peers { permanent_uuid: "b24f3658c5a146919e1ac01f6a2e6505" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 } } peers { permanent_uuid: "635815b906114ae1b5dd994f0a71a4e4" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } } }
I20260619 02:53:29.326763 32669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [sys.catalog]: This master's current role is: LEADER
W20260619 02:53:29.334771 32434 tablet.cc:2369] T 00000000000000000000000000000000 P 47ee2de45aa9473db93d0a81b527317a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:53:29.337576 32672 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:53:29.337925 32554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b24f3658c5a146919e1ac01f6a2e6505: No bootstrap required, opened a new log
I20260619 02:53:29.339823 32672 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:53:29.359632 32455 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 47ee2de45aa9473db93d0a81b527317a [term 1 FOLLOWER]: Refusing update from remote peer 635815b906114ae1b5dd994f0a71a4e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:53:29.360919 32669 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47ee2de45aa9473db93d0a81b527317a" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:53:29.357723 32554 raft_consensus.cc:348] T 00000000000000000000000000000000 P b24f3658c5a146919e1ac01f6a2e6505 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47ee2de45aa9473db93d0a81b527317a" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32875 } } peers { permanent_uuid: "b24f3658c5a146919e1ac01f6a2e6505" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 } } peers { permanent_uuid: "635815b906114ae1b5dd994f0a71a4e4" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } }
I20260619 02:53:29.363433 32554 raft_consensus.cc:374] T 00000000000000000000000000000000 P b24f3658c5a146919e1ac01f6a2e6505 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:53:29.363780 32554 raft_consensus.cc:729] T 00000000000000000000000000000000 P b24f3658c5a146919e1ac01f6a2e6505 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24f3658c5a146919e1ac01f6a2e6505, State: Initialized, Role: FOLLOWER
I20260619 02:53:29.368507 32554 consensus_queue.cc:260] T 00000000000000000000000000000000 P b24f3658c5a146919e1ac01f6a2e6505 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47ee2de45aa9473db93d0a81b527317a" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32875 } } peers { permanent_uuid: "b24f3658c5a146919e1ac01f6a2e6505" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 } } peers { permanent_uuid: "635815b906114ae1b5dd994f0a71a4e4" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } }
I20260619 02:53:29.379262 32675 sys_catalog.cc:455] T 00000000000000000000000000000000 P b24f3658c5a146919e1ac01f6a2e6505 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47ee2de45aa9473db93d0a81b527317a" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32875 } } peers { permanent_uuid: "b24f3658c5a146919e1ac01f6a2e6505" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 } } peers { permanent_uuid: "635815b906114ae1b5dd994f0a71a4e4" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } } }
I20260619 02:53:29.379758 32675 sys_catalog.cc:458] T 00000000000000000000000000000000 P b24f3658c5a146919e1ac01f6a2e6505 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:53:29.382928 32528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "635815b906114ae1b5dd994f0a71a4e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24f3658c5a146919e1ac01f6a2e6505"
I20260619 02:53:29.382335 32529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "635815b906114ae1b5dd994f0a71a4e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24f3658c5a146919e1ac01f6a2e6505" is_pre_election: true
I20260619 02:53:29.383751 32527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b24f3658c5a146919e1ac01f6a2e6505 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:53:29.383880 32635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47ee2de45aa9473db93d0a81b527317a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 635815b906114ae1b5dd994f0a71a4e4. Latest consensus state: current_term: 1 leader_uuid: "635815b906114ae1b5dd994f0a71a4e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47ee2de45aa9473db93d0a81b527317a" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32875 } } peers { permanent_uuid: "b24f3658c5a146919e1ac01f6a2e6505" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 } } peers { permanent_uuid: "635815b906114ae1b5dd994f0a71a4e4" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } } }
I20260619 02:53:29.384166 32635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47ee2de45aa9473db93d0a81b527317a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:53:29.387362 32527 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b24f3658c5a146919e1ac01f6a2e6505 [term 1 FOLLOWER]: Refusing update from remote peer 635815b906114ae1b5dd994f0a71a4e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:53:29.389415 32650 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24f3658c5a146919e1ac01f6a2e6505" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:53:29.391784 32528 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b24f3658c5a146919e1ac01f6a2e6505 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 635815b906114ae1b5dd994f0a71a4e4 in term 1.
I20260619 02:53:29.392233 32673 mvcc.cc:205] Tried to move back new op lower bound from 7298406847922331648 to 7298406847754887168. Current Snapshot: MvccSnapshot[applied={T|T < 7298406847922331648}]
I20260619 02:53:29.394192 32529 raft_consensus.cc:2362] T 00000000000000000000000000000000 P b24f3658c5a146919e1ac01f6a2e6505 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 635815b906114ae1b5dd994f0a71a4e4 in term 1. Re-sending same reply.
I20260619 02:53:29.394884 32674 mvcc.cc:205] Tried to move back new op lower bound from 7298406847922331648 to 7298406847754887168. Current Snapshot: MvccSnapshot[applied={T|T < 7298406847922331648}]
I20260619 02:53:29.397403 32650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "635815b906114ae1b5dd994f0a71a4e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47ee2de45aa9473db93d0a81b527317a" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32875 } } peers { permanent_uuid: "b24f3658c5a146919e1ac01f6a2e6505" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 } } peers { permanent_uuid: "635815b906114ae1b5dd994f0a71a4e4" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } } }
I20260619 02:53:29.397732 32650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [sys.catalog]: This master's current role is: LEADER
I20260619 02:53:29.399297 32672 catalog_manager.cc:1347] Generated new cluster ID: 816319ad906e475cba6d1daca08985d1
I20260619 02:53:29.399391 32672 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:53:29.400216 32635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47ee2de45aa9473db93d0a81b527317a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "635815b906114ae1b5dd994f0a71a4e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47ee2de45aa9473db93d0a81b527317a" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32875 } } peers { permanent_uuid: "b24f3658c5a146919e1ac01f6a2e6505" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 } } peers { permanent_uuid: "635815b906114ae1b5dd994f0a71a4e4" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } } }
I20260619 02:53:29.400465 32635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47ee2de45aa9473db93d0a81b527317a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:53:29.400652 32554 sys_catalog.cc:564] T 00000000000000000000000000000000 P b24f3658c5a146919e1ac01f6a2e6505 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:53:29.424160 32672 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:53:29.429973 32672 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:53:29.443439 32672 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4: Generated new TSK 0
I20260619 02:53:29.444676 32672 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:53:29.447655 32683 sys_catalog.cc:455] T 00000000000000000000000000000000 P b24f3658c5a146919e1ac01f6a2e6505 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "635815b906114ae1b5dd994f0a71a4e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47ee2de45aa9473db93d0a81b527317a" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32875 } } peers { permanent_uuid: "b24f3658c5a146919e1ac01f6a2e6505" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 } } peers { permanent_uuid: "635815b906114ae1b5dd994f0a71a4e4" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } } }
I20260619 02:53:29.448081 32683 sys_catalog.cc:458] T 00000000000000000000000000000000 P b24f3658c5a146919e1ac01f6a2e6505 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:53:29.449463 32675 sys_catalog.cc:455] T 00000000000000000000000000000000 P b24f3658c5a146919e1ac01f6a2e6505 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 635815b906114ae1b5dd994f0a71a4e4. Latest consensus state: current_term: 1 leader_uuid: "635815b906114ae1b5dd994f0a71a4e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47ee2de45aa9473db93d0a81b527317a" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32875 } } peers { permanent_uuid: "b24f3658c5a146919e1ac01f6a2e6505" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 } } peers { permanent_uuid: "635815b906114ae1b5dd994f0a71a4e4" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } } }
I20260619 02:53:29.449748 32675 sys_catalog.cc:458] T 00000000000000000000000000000000 P b24f3658c5a146919e1ac01f6a2e6505 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:53:29.449944 32650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "635815b906114ae1b5dd994f0a71a4e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47ee2de45aa9473db93d0a81b527317a" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 32875 } } peers { permanent_uuid: "b24f3658c5a146919e1ac01f6a2e6505" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 43329 } } peers { permanent_uuid: "635815b906114ae1b5dd994f0a71a4e4" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 44773 } } }
I20260619 02:53:29.450223 32650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 635815b906114ae1b5dd994f0a71a4e4 [sys.catalog]: This master's current role is: LEADER
I20260619 02:53:29.531071 32698 catalog_manager.cc:1259] Loaded cluster ID: 816319ad906e475cba6d1daca08985d1
I20260619 02:53:29.531188 32698 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b24f3658c5a146919e1ac01f6a2e6505: loading cluster ID for follower catalog manager: success
I20260619 02:53:29.535126 32698 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b24f3658c5a146919e1ac01f6a2e6505: acquiring CA information for follower catalog manager: success
I20260619 02:53:29.537766 32698 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b24f3658c5a146919e1ac01f6a2e6505: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:53:30.139147 32647 catalog_manager.cc:1259] Loaded cluster ID: 816319ad906e475cba6d1daca08985d1
I20260619 02:53:30.139254 32647 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 47ee2de45aa9473db93d0a81b527317a: loading cluster ID for follower catalog manager: success
I20260619 02:53:30.141561 32647 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 47ee2de45aa9473db93d0a81b527317a: acquiring CA information for follower catalog manager: success
I20260619 02:53:30.143416 32647 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 47ee2de45aa9473db93d0a81b527317a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:53:30.574440 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 32413
I20260619 02:53:30.575740 32475 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:53:30.684958 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 32413
I20260619 02:53:30.701432 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 32487
I20260619 02:53:30.702477 32549 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:53:30.806711 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 32487
I20260619 02:53:30.822741 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 32561
I20260619 02:53:30.823911 32623 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:53:30.931775 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 32561
2026-06-19T02:53:30Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4467 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:53:31 dist-test-slave-9mr0 krb5kdc[32705](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:53:31 dist-test-slave-9mr0 krb5kdc[32705](info): set up 2 sockets
Jun 19 02:53:31 dist-test-slave-9mr0 krb5kdc[32705](info): commencing operation
krb5kdc: starting...
W20260619 02:53:31.946952 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.979s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:53:32 dist-test-slave-9mr0 krb5kdc[32705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:53:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:53:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:53:32.161965 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:46741
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:43055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:46741
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:53:32.476887 32718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:32.477170 32718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:32.477267 32718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:53:32.484386 32718 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:53:32.484491 32718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:32.484548 32718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:32.484603 32718 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:53:32.484647 32718 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:53:32.494095 32718 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:43055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:46741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:46741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:32.495301 32718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:32.496487 32718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:53:32.507840 32724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:32.507861 32726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:32.508373 32723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:32.508690 32718 server_base.cc:1034] running on GCE node
I20260619 02:53:32.509397 32718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:32.510848 32718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:32.512066 32718 hybrid_clock.cc:648] HybridClock initialized: now 1781837612512009 us; error 49 us; skew 500 ppm
I20260619 02:53:32.512368 32718 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:32 dist-test-slave-9mr0 krb5kdc[32705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837612, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:32.515498 32718 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:53:32.517000 32718 webserver.cc:460] Webserver started at http://127.18.134.62:32879/ using document root <none> and password file <none>
I20260619 02:53:32.517652 32718 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:32.517777 32718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:53:32.518071 32718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:53:32.521039 32718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "69e3e0c190104acc8f8139b2bef685e2"
format_stamp: "Formatted at 2026-06-19 02:53:32 on dist-test-slave-9mr0"
server_key: "24b4a58cfb16dc9b895744682713b680"
server_key_iv: "9d6edea8197a1ace2d648250e6e2017d"
server_key_version: "encryptionkey@0"
I20260619 02:53:32.521764 32718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "69e3e0c190104acc8f8139b2bef685e2"
format_stamp: "Formatted at 2026-06-19 02:53:32 on dist-test-slave-9mr0"
server_key: "24b4a58cfb16dc9b895744682713b680"
server_key_iv: "9d6edea8197a1ace2d648250e6e2017d"
server_key_version: "encryptionkey@0"
I20260619 02:53:32.526775 32718 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260619 02:53:32.530058 32733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:32.531426 32718 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:53:32.531584 32718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "69e3e0c190104acc8f8139b2bef685e2"
format_stamp: "Formatted at 2026-06-19 02:53:32 on dist-test-slave-9mr0"
server_key: "24b4a58cfb16dc9b895744682713b680"
server_key_iv: "9d6edea8197a1ace2d648250e6e2017d"
server_key_version: "encryptionkey@0"
I20260619 02:53:32.531766 32718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:32.545495 32718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:32.549515 32718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:32.549772 32718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:32.566850 32718 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:46741
I20260619 02:53:32.566880 316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:46741 every 8 connection(s)
I20260619 02:53:32.568120 32718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:53:32.571714 317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:53:32.571942 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 32718
I20260619 02:53:32.572154 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:53:32.572937 18968 external_mini_cluster.cc:1428] Setting key 0e9e8fa6d13cf6b1a37d6e420d399caa
I20260619 02:53:32.584991 317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69e3e0c190104acc8f8139b2bef685e2: Bootstrap starting.
Jun 19 02:53:32 dist-test-slave-9mr0 krb5kdc[32705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:53:32.588706 317 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69e3e0c190104acc8f8139b2bef685e2: Neither blocks nor log segments found. Creating new log.
I20260619 02:53:32.589787 317 log.cc:824] T 00000000000000000000000000000000 P 69e3e0c190104acc8f8139b2bef685e2: Log is configured to *not* fsync() on all Append() calls
I20260619 02:53:32.592720 317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69e3e0c190104acc8f8139b2bef685e2: No bootstrap required, opened a new log
I20260619 02:53:32.596232 320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:32.574696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51078 (local address 127.18.134.62:46741)
0619 02:53:32.575467 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:32.575478 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:32.575516 (+ 38us) server_negotiation.cc:407] Connection header received
0619 02:53:32.577667 (+ 2151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:32.577687 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:32.578076 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:32.578442 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:32.580187 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:32.581466 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:32.583408 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:32.584085 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:32.584228 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:32.589063 (+ 4835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:32.589076 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:32.589136 (+ 60us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:32.589187 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:32.592281 (+ 3094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:32.593097 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:32.593103 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:32.593108 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:32.593277 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:32.593683 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:32.593686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:32.593688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:32.594185 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:32.594409 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:32.594869 (+ 460us) server_negotiation.cc:299] Negotiation successful
0619 02:53:32.595227 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":344,"threads_started":1}
I20260619 02:53:32.600198 18968 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:53:32.601965 317 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69e3e0c190104acc8f8139b2bef685e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e3e0c190104acc8f8139b2bef685e2" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 46741 } }
I20260619 02:53:32.602373 317 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69e3e0c190104acc8f8139b2bef685e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:53:32.602469 317 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69e3e0c190104acc8f8139b2bef685e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69e3e0c190104acc8f8139b2bef685e2, State: Initialized, Role: FOLLOWER
I20260619 02:53:32.603076 317 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69e3e0c190104acc8f8139b2bef685e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e3e0c190104acc8f8139b2bef685e2" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 46741 } }
I20260619 02:53:32.603303 317 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69e3e0c190104acc8f8139b2bef685e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:53:32.603435 317 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69e3e0c190104acc8f8139b2bef685e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:53:32.603583 317 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69e3e0c190104acc8f8139b2bef685e2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:53:32.605772 317 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69e3e0c190104acc8f8139b2bef685e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e3e0c190104acc8f8139b2bef685e2" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 46741 } }
I20260619 02:53:32.606222 317 leader_election.cc:302] T 00000000000000000000000000000000 P 69e3e0c190104acc8f8139b2bef685e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69e3e0c190104acc8f8139b2bef685e2; no voters:
I20260619 02:53:32.606797 317 leader_election.cc:288] T 00000000000000000000000000000000 P 69e3e0c190104acc8f8139b2bef685e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:53:32.606918 322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69e3e0c190104acc8f8139b2bef685e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:53:32.607540 322 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69e3e0c190104acc8f8139b2bef685e2 [term 1 LEADER]: Becoming Leader. State: Replica: 69e3e0c190104acc8f8139b2bef685e2, State: Running, Role: LEADER
I20260619 02:53:32.608023 322 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69e3e0c190104acc8f8139b2bef685e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e3e0c190104acc8f8139b2bef685e2" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 46741 } }
I20260619 02:53:32.608614 317 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69e3e0c190104acc8f8139b2bef685e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:53:32.610488 323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69e3e0c190104acc8f8139b2bef685e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69e3e0c190104acc8f8139b2bef685e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e3e0c190104acc8f8139b2bef685e2" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 46741 } } }
I20260619 02:53:32.610915 323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69e3e0c190104acc8f8139b2bef685e2 [sys.catalog]: This master's current role is: LEADER
I20260619 02:53:32.610880 324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69e3e0c190104acc8f8139b2bef685e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69e3e0c190104acc8f8139b2bef685e2. Latest consensus state: current_term: 1 leader_uuid: "69e3e0c190104acc8f8139b2bef685e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e3e0c190104acc8f8139b2bef685e2" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 46741 } } }
I20260619 02:53:32.611171 324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69e3e0c190104acc8f8139b2bef685e2 [sys.catalog]: This master's current role is: LEADER
I20260619 02:53:32.613240 330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:53:32.617290 330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:53:32.625034 330 catalog_manager.cc:1347] Generated new cluster ID: 66b74d55d5334f909f4369f24128941a
I20260619 02:53:32.625159 330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:53:32.651667 330 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:53:32.653029 330 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:53:32.659891 330 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69e3e0c190104acc8f8139b2bef685e2: Generated new TSK 0
I20260619 02:53:32.660665 330 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:53:32.713107 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 32718
I20260619 02:53:32.714351 312 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:53:32.809146 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 32718
2026-06-19T02:53:32Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1874 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:53:32 dist-test-slave-9mr0 krb5kdc[345](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:53:32 dist-test-slave-9mr0 krb5kdc[345](info): set up 2 sockets
Jun 19 02:53:32 dist-test-slave-9mr0 krb5kdc[345](info): commencing operation
krb5kdc: starting...
W20260619 02:53:33.818912 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.976s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:53:33 dist-test-slave-9mr0 krb5kdc[345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:53:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:53:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:53:34.036525 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:33207
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:33207
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:53:34.348304 358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:34.348606 358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:34.348750 358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:53:34.355949 358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:53:34.356076 358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:34.356154 358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:34.356236 358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:53:34.356297 358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:53:34.366050 358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:33207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:33207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:34.367412 358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:34.368650 358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:53:34.379731 363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:34.379805 366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:34.379765 364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:34.380625 358 server_base.cc:1034] running on GCE node
I20260619 02:53:34.381327 358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:34.382668 358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:34.383874 358 hybrid_clock.cc:648] HybridClock initialized: now 1781837614383822 us; error 39 us; skew 500 ppm
I20260619 02:53:34.384197 358 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:34 dist-test-slave-9mr0 krb5kdc[345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837614, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:34.387285 358 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:53:34.388777 358 webserver.cc:460] Webserver started at http://127.18.134.62:34703/ using document root <none> and password file <none>
I20260619 02:53:34.389443 358 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:34.389547 358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:53:34.389838 358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:53:34.392956 358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "6f92d2f13db7417da3983549b952dd91"
format_stamp: "Formatted at 2026-06-19 02:53:34 on dist-test-slave-9mr0"
server_key: "a666cd18c1b6770c6db96d1f4147971d"
server_key_iv: "8fa7a41f11bd84b00563b31988a41740"
server_key_version: "encryptionkey@0"
I20260619 02:53:34.393741 358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "6f92d2f13db7417da3983549b952dd91"
format_stamp: "Formatted at 2026-06-19 02:53:34 on dist-test-slave-9mr0"
server_key: "a666cd18c1b6770c6db96d1f4147971d"
server_key_iv: "8fa7a41f11bd84b00563b31988a41740"
server_key_version: "encryptionkey@0"
I20260619 02:53:34.399058 358 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260619 02:53:34.402612 373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:34.404095 358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:53:34.404264 358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "6f92d2f13db7417da3983549b952dd91"
format_stamp: "Formatted at 2026-06-19 02:53:34 on dist-test-slave-9mr0"
server_key: "a666cd18c1b6770c6db96d1f4147971d"
server_key_iv: "8fa7a41f11bd84b00563b31988a41740"
server_key_version: "encryptionkey@0"
I20260619 02:53:34.404457 358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:34.425202 358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:34.429468 358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:34.429728 358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:34.446907 358 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:33207
I20260619 02:53:34.446970 424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:33207 every 8 connection(s)
I20260619 02:53:34.448189 358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:53:34.452596 425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:53:34.456561 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 358
I20260619 02:53:34.456784 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:53:34.457527 18968 external_mini_cluster.cc:1428] Setting key 8c4ce732eb9c5d26479347356b6dbd37
I20260619 02:53:34.465974 425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91: Bootstrap starting.
I20260619 02:53:34.469916 425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91: Neither blocks nor log segments found. Creating new log.
I20260619 02:53:34.471351 425 log.cc:824] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91: Log is configured to *not* fsync() on all Append() calls
Jun 19 02:53:34 dist-test-slave-9mr0 krb5kdc[345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:53:34.474572 425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91: No bootstrap required, opened a new log
I20260619 02:53:34.482487 425 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f92d2f13db7417da3983549b952dd91" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33207 } }
I20260619 02:53:34.482951 425 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:53:34.483076 425 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f92d2f13db7417da3983549b952dd91, State: Initialized, Role: FOLLOWER
I20260619 02:53:34.483749 425 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f92d2f13db7417da3983549b952dd91" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33207 } }
I20260619 02:53:34.483963 425 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:53:34.484092 425 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:53:34.484262 425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:53:34.484865 428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:34.459507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37164 (local address 127.18.134.62:33207)
0619 02:53:34.460220 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:34.460233 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:34.460272 (+ 39us) server_negotiation.cc:407] Connection header received
0619 02:53:34.461810 (+ 1538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:34.461831 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:34.462235 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:34.462562 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:34.464401 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:34.465688 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:34.467516 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:34.468231 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:34.468371 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:34.473615 (+ 5244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:34.473623 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:34.473638 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:34.473673 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:34.480573 (+ 6900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:34.481435 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:34.481441 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:34.481448 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:34.481618 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:34.482031 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:34.482034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:34.482036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:34.482569 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:34.482853 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:34.483301 (+ 448us) server_negotiation.cc:299] Negotiation successful
0619 02:53:34.483739 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":325,"threads_started":1}
I20260619 02:53:34.486554 425 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f92d2f13db7417da3983549b952dd91" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33207 } }
I20260619 02:53:34.487067 425 leader_election.cc:302] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f92d2f13db7417da3983549b952dd91; no voters:
I20260619 02:53:34.487700 425 leader_election.cc:288] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:53:34.487821 430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:53:34.488446 430 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [term 1 LEADER]: Becoming Leader. State: Replica: 6f92d2f13db7417da3983549b952dd91, State: Running, Role: LEADER
I20260619 02:53:34.488976 430 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f92d2f13db7417da3983549b952dd91" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33207 } }
I20260619 02:53:34.489571 425 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:53:34.491464 431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f92d2f13db7417da3983549b952dd91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f92d2f13db7417da3983549b952dd91" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33207 } } }
I20260619 02:53:34.492002 432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f92d2f13db7417da3983549b952dd91. Latest consensus state: current_term: 1 leader_uuid: "6f92d2f13db7417da3983549b952dd91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f92d2f13db7417da3983549b952dd91" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33207 } } }
I20260619 02:53:34.492396 431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [sys.catalog]: This master's current role is: LEADER
I20260619 02:53:34.492478 432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [sys.catalog]: This master's current role is: LEADER
I20260619 02:53:34.495139 437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:53:34.499434 437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:53:34.508026 437 catalog_manager.cc:1347] Generated new cluster ID: c8aa49fb650f439e845d95a09df7e6f9
I20260619 02:53:34.508152 437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:53:34.519958 437 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:53:34.521788 437 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:53:34.529968 437 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91: Generated new TSK 0
I20260619 02:53:34.530815 437 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.134.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
I20260619 02:53:34.594822 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:0
--local_ip_for_outbound_sockets=127.18.134.1
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:33207
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:53:34.924132 451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:34.924412 451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:34.924551 451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:53:34.931715 451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:34.931842 451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:34.931988 451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:53:34.941672 451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:33207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:34.943005 451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:34.944267 451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:53:34.956619 456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:34.956665 459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:34.956645 457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:34.957361 451 server_base.cc:1034] running on GCE node
I20260619 02:53:34.957978 451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:34.958962 451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:34.960145 451 hybrid_clock.cc:648] HybridClock initialized: now 1781837614960090 us; error 53 us; skew 500 ppm
I20260619 02:53:34.960435 451 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:34 dist-test-slave-9mr0 krb5kdc[345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837614, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:34.963459 451 init.cc:377] Logged in from keytab as kudu/127.18.134.1@KRBTEST.COM (short username kudu)
I20260619 02:53:34.964823 451 webserver.cc:460] Webserver started at http://127.18.134.1:36467/ using document root <none> and password file <none>
I20260619 02:53:34.965487 451 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:34.965602 451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:53:34.965883 451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:53:34.968715 451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/data/instance:
uuid: "8eb61eb911a24442acb8c3662c346f80"
format_stamp: "Formatted at 2026-06-19 02:53:34 on dist-test-slave-9mr0"
server_key: "650df52c8f80a9b31296004d51d0b22a"
server_key_iv: "2632c9cb4bc69fab6feab44663ff1d34"
server_key_version: "encryptionkey@0"
I20260619 02:53:34.969478 451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance:
uuid: "8eb61eb911a24442acb8c3662c346f80"
format_stamp: "Formatted at 2026-06-19 02:53:34 on dist-test-slave-9mr0"
server_key: "650df52c8f80a9b31296004d51d0b22a"
server_key_iv: "2632c9cb4bc69fab6feab44663ff1d34"
server_key_version: "encryptionkey@0"
I20260619 02:53:34.974534 451 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 02:53:34.977931 466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:34.979447 451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260619 02:53:34.979610 451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "8eb61eb911a24442acb8c3662c346f80"
format_stamp: "Formatted at 2026-06-19 02:53:34 on dist-test-slave-9mr0"
server_key: "650df52c8f80a9b31296004d51d0b22a"
server_key_iv: "2632c9cb4bc69fab6feab44663ff1d34"
server_key_version: "encryptionkey@0"
I20260619 02:53:34.979820 451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:35.004006 451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:35.008414 451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:35.008674 451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:35.009640 451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:53:35.010972 451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:53:35.011051 451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:35.011171 451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:53:35.011225 451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:35.040823 451 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:34647
I20260619 02:53:35.040887 578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:34647 every 8 connection(s)
I20260619 02:53:35.042028 451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:53:35.048115 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 451
I20260619 02:53:35.048295 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance
I20260619 02:53:35.048899 18968 external_mini_cluster.cc:1428] Setting key 4f27df06a5aa839938bc2a677bfa9800
Jun 19 02:53:35 dist-test-slave-9mr0 krb5kdc[345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837614, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:53:35.064363 582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:35.044810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.1:40183)
0619 02:53:35.045512 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:35.045554 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:35.047206 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:35.047903 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:35.047911 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:35.048473 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:53:35.049036 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:35.049055 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:35.050181 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:35.050185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:35.051541 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:35.051554 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:35.052301 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:35.052308 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:35.052513 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:35.053281 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:53:35.053352 (+ 71us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:53:35.055978 (+ 2626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:53:35.059858 (+ 3880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:35.059868 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:35.059884 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:35.060483 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:35.060911 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:35.060916 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:35.060920 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:35.061149 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:35.061766 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:53:35.061774 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:53:35.062217 (+ 443us) client_negotiation.cc:769] Sending connection context
0619 02:53:35.062478 (+ 261us) client_negotiation.cc:240] Negotiation successful
0619 02:53:35.063036 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":266,"threads_started":1}
I20260619 02:53:35.065562 581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:35.044393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:40183 (local address 127.18.134.62:33207)
0619 02:53:35.044831 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:35.044835 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:35.045661 (+ 826us) server_negotiation.cc:407] Connection header received
0619 02:53:35.047460 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:35.047466 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:35.047555 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:35.047649 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:35.049190 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:35.050021 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:35.051683 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:35.052082 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:35.052192 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:35.056253 (+ 4061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:35.056260 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:35.056264 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:35.056302 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:35.059711 (+ 3409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:35.060638 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:35.060643 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:35.060647 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:35.060794 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:35.061309 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:35.061313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:35.061317 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:35.061624 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:35.064823 (+ 3199us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:35.064963 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 02:53:35.065215 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":215,"threads_started":1}
I20260619 02:53:35.066764 579 heartbeater.cc:347] Connected to a master server at 127.18.134.62:33207
I20260619 02:53:35.067078 579 heartbeater.cc:464] Registering TS with master...
I20260619 02:53:35.067924 579 heartbeater.cc:511] Master 127.18.134.62:33207 requested a full tablet report, sending...
I20260619 02:53:35.070073 390 ts_manager.cc:194] Registered new tserver with Master: 8eb61eb911a24442acb8c3662c346f80 (127.18.134.1:34647)
I20260619 02:53:35.071544 390 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.1@KRBTEST.COM'} at 127.18.134.1:40183
WARNING: no policy specified for kudu/127.18.134.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
I20260619 02:53:35.099702 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:0
--local_ip_for_outbound_sockets=127.18.134.2
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:33207
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:53:35.409148 585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:35.409449 585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:35.409548 585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:53:35.416710 585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:35.416818 585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:35.416926 585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:53:35.426704 585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:33207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:35.427960 585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:35.429204 585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:53:35.441871 591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:35.441840 590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:35.441807 593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:35.442611 585 server_base.cc:1034] running on GCE node
I20260619 02:53:35.443182 585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:35.444023 585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:35.445231 585 hybrid_clock.cc:648] HybridClock initialized: now 1781837615445175 us; error 46 us; skew 500 ppm
I20260619 02:53:35.445499 585 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:35 dist-test-slave-9mr0 krb5kdc[345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837615, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:35.448727 585 init.cc:377] Logged in from keytab as kudu/127.18.134.2@KRBTEST.COM (short username kudu)
I20260619 02:53:35.450286 585 webserver.cc:460] Webserver started at http://127.18.134.2:42729/ using document root <none> and password file <none>
I20260619 02:53:35.451089 585 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:35.451225 585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:53:35.451561 585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:53:35.455253 585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/data/instance:
uuid: "011f109736ca46a0b59476e947fc20a3"
format_stamp: "Formatted at 2026-06-19 02:53:35 on dist-test-slave-9mr0"
server_key: "794a5cd224ed315bd923c3e7fdcde710"
server_key_iv: "76c8d3f264a2dc32f478810894caf48e"
server_key_version: "encryptionkey@0"
I20260619 02:53:35.456152 585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance:
uuid: "011f109736ca46a0b59476e947fc20a3"
format_stamp: "Formatted at 2026-06-19 02:53:35 on dist-test-slave-9mr0"
server_key: "794a5cd224ed315bd923c3e7fdcde710"
server_key_iv: "76c8d3f264a2dc32f478810894caf48e"
server_key_version: "encryptionkey@0"
I20260619 02:53:35.462195 585 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260619 02:53:35.466382 600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:35.467698 585 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:53:35.467839 585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "011f109736ca46a0b59476e947fc20a3"
format_stamp: "Formatted at 2026-06-19 02:53:35 on dist-test-slave-9mr0"
server_key: "794a5cd224ed315bd923c3e7fdcde710"
server_key_iv: "76c8d3f264a2dc32f478810894caf48e"
server_key_version: "encryptionkey@0"
I20260619 02:53:35.468001 585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:35.490967 585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:35.494827 585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:35.495052 585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:35.495944 585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:53:35.497182 585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:53:35.497252 585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:35.497347 585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:53:35.497390 585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:35.526477 585 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:43259
I20260619 02:53:35.526525 712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:43259 every 8 connection(s)
I20260619 02:53:35.527773 585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:53:35.529739 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 585
I20260619 02:53:35.529886 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance
I20260619 02:53:35.530417 18968 external_mini_cluster.cc:1428] Setting key 536076f80ec71b71f309e9cdd7e7cd3a
Jun 19 02:53:35 dist-test-slave-9mr0 krb5kdc[345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837615, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:53:35.549551 581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:35.530242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:39313 (local address 127.18.134.62:33207)
0619 02:53:35.530443 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:35.530449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:35.531762 (+ 1313us) server_negotiation.cc:407] Connection header received
0619 02:53:35.534030 (+ 2268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:35.534035 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:35.534100 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:35.534172 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:35.536102 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:35.536887 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:35.538984 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:35.539378 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:35.539475 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:35.543542 (+ 4067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:35.543549 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:35.543554 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:35.543594 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:35.546493 (+ 2899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:35.547269 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:35.547274 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:35.547277 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:35.547437 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:35.547822 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:35.547827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:35.547830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:35.548163 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:35.548291 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:35.548977 (+ 686us) server_negotiation.cc:299] Negotiation successful
0619 02:53:35.549206 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 02:53:35.550336 715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:35.530808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.2:39313)
0619 02:53:35.531618 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:35.531656 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:35.533729 (+ 2073us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:35.534432 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:35.534445 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:35.535157 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:53:35.535904 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:35.535931 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:35.537042 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:35.537048 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:35.538853 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:35.538868 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:35.539579 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:35.539585 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:35.539862 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:35.540573 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:53:35.540603 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:53:35.543259 (+ 2656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:53:35.546698 (+ 3439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:35.546706 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:35.546719 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:35.547139 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:35.547527 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:35.547530 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:35.547532 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:35.547723 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:35.548283 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:53:35.548288 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:53:35.548600 (+ 312us) client_negotiation.cc:769] Sending connection context
0619 02:53:35.548846 (+ 246us) client_negotiation.cc:240] Negotiation successful
0619 02:53:35.549247 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":327,"threads_started":1}
I20260619 02:53:35.551868 713 heartbeater.cc:347] Connected to a master server at 127.18.134.62:33207
I20260619 02:53:35.552232 713 heartbeater.cc:464] Registering TS with master...
I20260619 02:53:35.553112 713 heartbeater.cc:511] Master 127.18.134.62:33207 requested a full tablet report, sending...
I20260619 02:53:35.554700 390 ts_manager.cc:194] Registered new tserver with Master: 011f109736ca46a0b59476e947fc20a3 (127.18.134.2:43259)
WARNING: no policy specified for kudu/127.18.134.3@KRBTEST.COM; defaulting to no policy
I20260619 02:53:35.555496 390 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.2@KRBTEST.COM'} at 127.18.134.2:39313
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
I20260619 02:53:35.581456 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:0
--local_ip_for_outbound_sockets=127.18.134.3
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:33207
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:53:35.890080 718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:35.890405 718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:35.890550 718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:53:35.897940 718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:35.898065 718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:35.898226 718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:53:35.908099 718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:33207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:35.909396 718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:35.910742 718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:53:35.923614 723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:35.923779 726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:35.923787 724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:35.924878 718 server_base.cc:1034] running on GCE node
I20260619 02:53:35.925558 718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:35.926563 718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:35.927820 718 hybrid_clock.cc:648] HybridClock initialized: now 1781837615927754 us; error 54 us; skew 500 ppm
I20260619 02:53:35.928148 718 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:35 dist-test-slave-9mr0 krb5kdc[345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837615, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:35.931334 718 init.cc:377] Logged in from keytab as kudu/127.18.134.3@KRBTEST.COM (short username kudu)
I20260619 02:53:35.932983 718 webserver.cc:460] Webserver started at http://127.18.134.3:43619/ using document root <none> and password file <none>
I20260619 02:53:35.933668 718 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:35.933785 718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:53:35.934095 718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:53:35.937141 718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/data/instance:
uuid: "083e444232ef4cc2a26fbee90b70b87d"
format_stamp: "Formatted at 2026-06-19 02:53:35 on dist-test-slave-9mr0"
server_key: "582892f9e0421d7d602e1303a4c62d90"
server_key_iv: "194c56670a55ce2eaf708d1617bac467"
server_key_version: "encryptionkey@0"
I20260619 02:53:35.937947 718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance:
uuid: "083e444232ef4cc2a26fbee90b70b87d"
format_stamp: "Formatted at 2026-06-19 02:53:35 on dist-test-slave-9mr0"
server_key: "582892f9e0421d7d602e1303a4c62d90"
server_key_iv: "194c56670a55ce2eaf708d1617bac467"
server_key_version: "encryptionkey@0"
I20260619 02:53:35.943230 718 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260619 02:53:35.946851 733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:35.948163 718 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:53:35.948321 718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "083e444232ef4cc2a26fbee90b70b87d"
format_stamp: "Formatted at 2026-06-19 02:53:35 on dist-test-slave-9mr0"
server_key: "582892f9e0421d7d602e1303a4c62d90"
server_key_iv: "194c56670a55ce2eaf708d1617bac467"
server_key_version: "encryptionkey@0"
I20260619 02:53:35.948515 718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:35.976740 718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:35.980940 718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:35.981204 718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:35.982101 718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:53:35.983470 718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:53:35.983541 718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:35.983651 718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:53:35.983702 718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:36.013072 718 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:44933
I20260619 02:53:36.013110 845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:44933 every 8 connection(s)
I20260619 02:53:36.014267 718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:53:36.021111 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 718
I20260619 02:53:36.021268 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance
I20260619 02:53:36.021677 18968 external_mini_cluster.cc:1428] Setting key 7202b8d3ca6837574a0439298eec07ba
Jun 19 02:53:36 dist-test-slave-9mr0 krb5kdc[345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837615, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:53:36.040225 581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:36.016719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:54765 (local address 127.18.134.62:33207)
0619 02:53:36.016931 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:36.016936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:36.018040 (+ 1104us) server_negotiation.cc:407] Connection header received
0619 02:53:36.019845 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:36.019851 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:36.019953 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:36.020040 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:36.021529 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.022316 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.024118 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.024588 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.024684 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:36.034675 (+ 9991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:36.034681 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:36.034684 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:36.034719 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:36.037438 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:36.038116 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:36.038120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:36.038122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:36.038244 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:36.038711 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:36.038714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:36.038716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:36.038994 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:36.039101 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:36.039763 (+ 662us) server_negotiation.cc:299] Negotiation successful
0619 02:53:36.039940 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260619 02:53:36.041172 848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:36.017161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.3:54765)
0619 02:53:36.017872 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:36.017918 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:36.019586 (+ 1668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:36.020275 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:36.020283 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:36.020775 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:53:36.021363 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:36.021388 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.022460 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.022465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:36.023991 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:36.024006 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.024788 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.024794 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:36.024994 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:36.032147 (+ 7153us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:53:36.032173 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:53:36.034417 (+ 2244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:53:36.037566 (+ 3149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:36.037573 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:36.037587 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:36.038023 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:36.038364 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:36.038368 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:36.038371 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:36.038557 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:36.039103 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:53:36.039109 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:53:36.039453 (+ 344us) client_negotiation.cc:769] Sending connection context
0619 02:53:36.039673 (+ 220us) client_negotiation.cc:240] Negotiation successful
0619 02:53:36.040080 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":277,"threads_started":1}
I20260619 02:53:36.042542 846 heartbeater.cc:347] Connected to a master server at 127.18.134.62:33207
I20260619 02:53:36.042833 846 heartbeater.cc:464] Registering TS with master...
I20260619 02:53:36.043519 846 heartbeater.cc:511] Master 127.18.134.62:33207 requested a full tablet report, sending...
I20260619 02:53:36.044898 390 ts_manager.cc:194] Registered new tserver with Master: 083e444232ef4cc2a26fbee90b70b87d (127.18.134.3:44933)
I20260619 02:53:36.045661 390 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.3@KRBTEST.COM'} at 127.18.134.3:54765
I20260619 02:53:36.049002 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:53:36.064865 581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:36.053242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37172 (local address 127.18.134.62:33207)
0619 02:53:36.053433 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:36.053440 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:36.053790 (+ 350us) server_negotiation.cc:407] Connection header received
0619 02:53:36.053911 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:36.053915 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:36.053976 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:36.054049 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:36.055389 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.056214 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.057455 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.057879 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.057977 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:36.060216 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:36.060221 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:36.060224 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:36.060261 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:36.062718 (+ 2457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:36.063274 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:36.063278 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:36.063280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:36.063409 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:36.063680 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:36.063682 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:36.063684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:36.063992 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:36.064172 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:36.064400 (+ 228us) server_negotiation.cc:299] Negotiation successful
0619 02:53:36.064556 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 02:53:36.068434 390 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37172:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 02:53:36.074702 579 heartbeater.cc:503] Master 127.18.134.62:33207 was elected leader, sending a full tablet report...
W20260619 02:53:36.075486 390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:53:36.114544 858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:36.086964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36002 (local address 127.18.134.1:34647)
0619 02:53:36.103057 (+ 16093us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:36.103066 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:36.104753 (+ 1687us) server_negotiation.cc:407] Connection header received
0619 02:53:36.105154 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:36.105172 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:36.105444 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:36.105658 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:53:36.106724 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.108197 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.112015 (+ 3818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.113103 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.113246 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:36.113459 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:36.114091 (+ 632us) server_negotiation.cc:299] Negotiation successful
0619 02:53:36.114243 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15918,"thread_start_us":15756,"threads_started":1}
I20260619 02:53:36.114578 861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:36.086959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:34647 (local address 127.0.0.1:36002)
0619 02:53:36.104558 (+ 17599us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:36.104610 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:36.104853 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:36.106026 (+ 1173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:36.106034 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:36.106063 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:53:36.106512 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:36.106524 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.108322 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.108328 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:36.111874 (+ 3546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:36.111884 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.113593 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.113599 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:36.113884 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:36.113918 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 02:53:36.114000 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 02:53:36.114191 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17302,"mutex_wait_us":1732,"thread_start_us":18825,"threads_started":1}
I20260619 02:53:36.114589 863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:36.087907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57556 (local address 127.18.134.3:44933)
0619 02:53:36.103944 (+ 16037us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:36.103953 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:36.106112 (+ 2159us) server_negotiation.cc:407] Connection header received
0619 02:53:36.106473 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:36.106481 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:36.106733 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:36.106898 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:53:36.107791 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.109285 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.111503 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.112915 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.113033 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:36.113265 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:36.114091 (+ 826us) server_negotiation.cc:299] Negotiation successful
0619 02:53:36.114256 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15863,"thread_start_us":15704,"threads_started":1}
I20260619 02:53:36.114598 862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:36.087690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56806 (local address 127.18.134.2:43259)
0619 02:53:36.103412 (+ 15722us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:36.103422 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:36.105610 (+ 2188us) server_negotiation.cc:407] Connection header received
0619 02:53:36.106386 (+ 776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:36.106394 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:36.106655 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:36.106797 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:53:36.107550 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.109176 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.111445 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.112898 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.113032 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:36.113235 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:36.114133 (+ 898us) server_negotiation.cc:299] Negotiation successful
0619 02:53:36.114283 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15545,"thread_start_us":15421,"threads_started":1}
I20260619 02:53:36.115345 859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:36.087330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:44933 (local address 127.0.0.1:57556)
0619 02:53:36.106005 (+ 18675us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:36.106018 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:36.106231 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:36.107188 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:36.107193 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:36.107217 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:53:36.107622 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:36.107633 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.109433 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.109438 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:36.111370 (+ 1932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:36.111384 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.113447 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.113454 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:36.113715 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:36.113845 (+ 130us) client_negotiation.cc:769] Sending connection context
0619 02:53:36.113972 (+ 127us) client_negotiation.cc:240] Negotiation successful
0619 02:53:36.114972 (+ 1000us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18573,"mutex_wait_us":146,"thread_start_us":16802,"threads_started":1}
I20260619 02:53:36.115595 860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:36.087231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:43259 (local address 127.0.0.1:56806)
0619 02:53:36.105491 (+ 18260us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:36.105509 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:36.106179 (+ 670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:36.107006 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:36.107009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:36.107028 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:53:36.107377 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:36.107388 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.109334 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.109339 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:36.111293 (+ 1954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:36.111307 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.113409 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.113417 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:36.113715 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:36.113868 (+ 153us) client_negotiation.cc:769] Sending connection context
0619 02:53:36.113972 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 02:53:36.115208 (+ 1236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18108,"mutex_wait_us":1316,"thread_start_us":17948,"threads_started":1}
I20260619 02:53:36.118876 514 tablet_service.cc:1467] Processing CreateTablet for tablet 5a5a01e8c36a4a558f96fe13fb3ae526 (DEFAULT_TABLE table=test-table [id=973a6bdbf8d74f9095dff465e60e293a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:53:36.118876 648 tablet_service.cc:1467] Processing CreateTablet for tablet 5a5a01e8c36a4a558f96fe13fb3ae526 (DEFAULT_TABLE table=test-table [id=973a6bdbf8d74f9095dff465e60e293a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:53:36.118883 781 tablet_service.cc:1467] Processing CreateTablet for tablet 5a5a01e8c36a4a558f96fe13fb3ae526 (DEFAULT_TABLE table=test-table [id=973a6bdbf8d74f9095dff465e60e293a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:53:36.120136 781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a5a01e8c36a4a558f96fe13fb3ae526. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:53:36.120234 514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a5a01e8c36a4a558f96fe13fb3ae526. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:53:36.120234 648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a5a01e8c36a4a558f96fe13fb3ae526. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:53:36.131320 864 tablet_bootstrap.cc:492] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d: Bootstrap starting.
I20260619 02:53:36.134281 866 tablet_bootstrap.cc:492] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 8eb61eb911a24442acb8c3662c346f80: Bootstrap starting.
I20260619 02:53:36.134807 864 tablet_bootstrap.cc:654] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d: Neither blocks nor log segments found. Creating new log.
I20260619 02:53:36.135521 865 tablet_bootstrap.cc:492] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 011f109736ca46a0b59476e947fc20a3: Bootstrap starting.
I20260619 02:53:36.136189 864 log.cc:824] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d: Log is configured to *not* fsync() on all Append() calls
I20260619 02:53:36.139107 865 tablet_bootstrap.cc:654] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 011f109736ca46a0b59476e947fc20a3: Neither blocks nor log segments found. Creating new log.
I20260619 02:53:36.140146 864 tablet_bootstrap.cc:492] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d: No bootstrap required, opened a new log
I20260619 02:53:36.140201 866 tablet_bootstrap.cc:654] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 8eb61eb911a24442acb8c3662c346f80: Neither blocks nor log segments found. Creating new log.
I20260619 02:53:36.140311 865 log.cc:824] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 011f109736ca46a0b59476e947fc20a3: Log is configured to *not* fsync() on all Append() calls
I20260619 02:53:36.140434 864 ts_tablet_manager.cc:1397] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.005s
I20260619 02:53:36.141315 866 log.cc:824] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 8eb61eb911a24442acb8c3662c346f80: Log is configured to *not* fsync() on all Append() calls
I20260619 02:53:36.143263 865 tablet_bootstrap.cc:492] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 011f109736ca46a0b59476e947fc20a3: No bootstrap required, opened a new log
I20260619 02:53:36.143540 865 ts_tablet_manager.cc:1397] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 011f109736ca46a0b59476e947fc20a3: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260619 02:53:36.144134 866 tablet_bootstrap.cc:492] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 8eb61eb911a24442acb8c3662c346f80: No bootstrap required, opened a new log
I20260619 02:53:36.144438 866 ts_tablet_manager.cc:1397] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 8eb61eb911a24442acb8c3662c346f80: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260619 02:53:36.148382 864 raft_consensus.cc:348] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb61eb911a24442acb8c3662c346f80" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 34647 } } peers { permanent_uuid: "011f109736ca46a0b59476e947fc20a3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43259 } } peers { permanent_uuid: "083e444232ef4cc2a26fbee90b70b87d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44933 } }
I20260619 02:53:36.148733 864 raft_consensus.cc:374] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:53:36.148844 864 raft_consensus.cc:729] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 083e444232ef4cc2a26fbee90b70b87d, State: Initialized, Role: FOLLOWER
I20260619 02:53:36.149428 864 consensus_queue.cc:260] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb61eb911a24442acb8c3662c346f80" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 34647 } } peers { permanent_uuid: "011f109736ca46a0b59476e947fc20a3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43259 } } peers { permanent_uuid: "083e444232ef4cc2a26fbee90b70b87d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44933 } }
I20260619 02:53:36.155041 865 raft_consensus.cc:348] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 011f109736ca46a0b59476e947fc20a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb61eb911a24442acb8c3662c346f80" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 34647 } } peers { permanent_uuid: "011f109736ca46a0b59476e947fc20a3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43259 } } peers { permanent_uuid: "083e444232ef4cc2a26fbee90b70b87d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44933 } }
I20260619 02:53:36.155625 865 raft_consensus.cc:374] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 011f109736ca46a0b59476e947fc20a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:53:36.155766 865 raft_consensus.cc:729] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 011f109736ca46a0b59476e947fc20a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 011f109736ca46a0b59476e947fc20a3, State: Initialized, Role: FOLLOWER
I20260619 02:53:36.156981 865 consensus_queue.cc:260] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 011f109736ca46a0b59476e947fc20a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb61eb911a24442acb8c3662c346f80" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 34647 } } peers { permanent_uuid: "011f109736ca46a0b59476e947fc20a3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43259 } } peers { permanent_uuid: "083e444232ef4cc2a26fbee90b70b87d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44933 } }
I20260619 02:53:36.159067 713 heartbeater.cc:503] Master 127.18.134.62:33207 was elected leader, sending a full tablet report...
I20260619 02:53:36.160264 865 ts_tablet_manager.cc:1428] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 011f109736ca46a0b59476e947fc20a3: Time spent starting tablet: real 0.017s user 0.010s sys 0.004s
I20260619 02:53:36.161365 846 heartbeater.cc:503] Master 127.18.134.62:33207 was elected leader, sending a full tablet report...
I20260619 02:53:36.162961 866 raft_consensus.cc:348] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 8eb61eb911a24442acb8c3662c346f80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb61eb911a24442acb8c3662c346f80" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 34647 } } peers { permanent_uuid: "011f109736ca46a0b59476e947fc20a3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43259 } } peers { permanent_uuid: "083e444232ef4cc2a26fbee90b70b87d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44933 } }
I20260619 02:53:36.163475 866 raft_consensus.cc:374] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 8eb61eb911a24442acb8c3662c346f80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:53:36.163620 866 raft_consensus.cc:729] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 8eb61eb911a24442acb8c3662c346f80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eb61eb911a24442acb8c3662c346f80, State: Initialized, Role: FOLLOWER
I20260619 02:53:36.163836 864 ts_tablet_manager.cc:1428] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d: Time spent starting tablet: real 0.023s user 0.011s sys 0.004s
I20260619 02:53:36.164381 866 consensus_queue.cc:260] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 8eb61eb911a24442acb8c3662c346f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb61eb911a24442acb8c3662c346f80" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 34647 } } peers { permanent_uuid: "011f109736ca46a0b59476e947fc20a3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43259 } } peers { permanent_uuid: "083e444232ef4cc2a26fbee90b70b87d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44933 } }
I20260619 02:53:36.167285 866 ts_tablet_manager.cc:1428] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 8eb61eb911a24442acb8c3662c346f80: Time spent starting tablet: real 0.023s user 0.008s sys 0.005s
I20260619 02:53:36.190488 870 raft_consensus.cc:482] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:53:36.190941 870 raft_consensus.cc:504] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb61eb911a24442acb8c3662c346f80" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 34647 } } peers { permanent_uuid: "011f109736ca46a0b59476e947fc20a3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43259 } } peers { permanent_uuid: "083e444232ef4cc2a26fbee90b70b87d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44933 } }
I20260619 02:53:36.193125 870 leader_election.cc:288] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eb61eb911a24442acb8c3662c346f80 (127.18.134.1:34647), 011f109736ca46a0b59476e947fc20a3 (127.18.134.2:43259)
I20260619 02:53:36.200196 848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:36.193863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:34647 (local address 127.18.134.3:48217)
0619 02:53:36.194096 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:36.194117 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:36.194252 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:36.194712 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:36.194716 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:36.194746 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:53:36.194961 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:36.194972 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.196163 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.196169 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:36.197785 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:36.197811 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.199444 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.199451 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:36.199639 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:36.199675 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 02:53:36.199771 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 02:53:36.199901 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260619 02:53:36.200210 858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:36.193491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:48217 (local address 127.18.134.1:34647)
0619 02:53:36.193676 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:36.193682 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:36.194415 (+ 733us) server_negotiation.cc:407] Connection header received
0619 02:53:36.194495 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:36.194500 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:36.194574 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:36.194691 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:53:36.195095 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.196054 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.197983 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.199292 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.199460 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:36.199540 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:36.199810 (+ 270us) server_negotiation.cc:299] Negotiation successful
0619 02:53:36.199949 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 02:53:36.201381 862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:36.193470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:37991 (local address 127.18.134.2:43259)
0619 02:53:36.193674 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:36.193681 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:36.194735 (+ 1054us) server_negotiation.cc:407] Connection header received
0619 02:53:36.195015 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:36.195021 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:36.195099 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:36.195184 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:53:36.195809 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.196896 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.198935 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.200028 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.200166 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:36.200234 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:36.201005 (+ 771us) server_negotiation.cc:299] Negotiation successful
0619 02:53:36.201139 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 02:53:36.201484 873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:36.193863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:43259 (local address 127.18.134.3:37991)
0619 02:53:36.194607 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:36.194622 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:36.194838 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:36.195206 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:36.195211 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:36.195236 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:53:36.195630 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:36.195641 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.197043 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.197048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:36.198790 (+ 1742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:36.198803 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.200636 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.200642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:36.200803 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:36.200835 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 02:53:36.200958 (+ 123us) client_negotiation.cc:240] Negotiation successful
0619 02:53:36.201078 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"mutex_wait_us":34,"thread_start_us":411,"threads_started":1}
I20260619 02:53:36.201398 534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a5a01e8c36a4a558f96fe13fb3ae526" candidate_uuid: "083e444232ef4cc2a26fbee90b70b87d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb61eb911a24442acb8c3662c346f80" is_pre_election: true
I20260619 02:53:36.202019 534 raft_consensus.cc:2454] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 8eb61eb911a24442acb8c3662c346f80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 083e444232ef4cc2a26fbee90b70b87d in term 0.
I20260619 02:53:36.202334 668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a5a01e8c36a4a558f96fe13fb3ae526" candidate_uuid: "083e444232ef4cc2a26fbee90b70b87d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "011f109736ca46a0b59476e947fc20a3" is_pre_election: true
I20260619 02:53:36.202849 668 raft_consensus.cc:2454] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 011f109736ca46a0b59476e947fc20a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 083e444232ef4cc2a26fbee90b70b87d in term 0.
I20260619 02:53:36.202879 737 leader_election.cc:302] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 083e444232ef4cc2a26fbee90b70b87d, 8eb61eb911a24442acb8c3662c346f80; no voters:
I20260619 02:53:36.203298 870 raft_consensus.cc:2781] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:53:36.203495 870 raft_consensus.cc:482] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:53:36.203609 870 raft_consensus.cc:3037] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:53:36.207019 870 raft_consensus.cc:504] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb61eb911a24442acb8c3662c346f80" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 34647 } } peers { permanent_uuid: "011f109736ca46a0b59476e947fc20a3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43259 } } peers { permanent_uuid: "083e444232ef4cc2a26fbee90b70b87d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44933 } }
I20260619 02:53:36.207789 870 leader_election.cc:288] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d [CANDIDATE]: Term 1 election: Requested vote from peers 8eb61eb911a24442acb8c3662c346f80 (127.18.134.1:34647), 011f109736ca46a0b59476e947fc20a3 (127.18.134.2:43259)
I20260619 02:53:36.208228 534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a5a01e8c36a4a558f96fe13fb3ae526" candidate_uuid: "083e444232ef4cc2a26fbee90b70b87d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb61eb911a24442acb8c3662c346f80"
I20260619 02:53:36.208297 668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a5a01e8c36a4a558f96fe13fb3ae526" candidate_uuid: "083e444232ef4cc2a26fbee90b70b87d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "011f109736ca46a0b59476e947fc20a3"
I20260619 02:53:36.208432 534 raft_consensus.cc:3037] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 8eb61eb911a24442acb8c3662c346f80 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:53:36.208518 668 raft_consensus.cc:3037] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 011f109736ca46a0b59476e947fc20a3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:53:36.210772 534 raft_consensus.cc:2454] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 8eb61eb911a24442acb8c3662c346f80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 083e444232ef4cc2a26fbee90b70b87d in term 1.
I20260619 02:53:36.211195 737 leader_election.cc:302] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 083e444232ef4cc2a26fbee90b70b87d, 8eb61eb911a24442acb8c3662c346f80; no voters:
I20260619 02:53:36.211792 668 raft_consensus.cc:2454] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 011f109736ca46a0b59476e947fc20a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 083e444232ef4cc2a26fbee90b70b87d in term 1.
I20260619 02:53:36.212142 870 raft_consensus.cc:2781] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:53:36.212623 870 raft_consensus.cc:686] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d [term 1 LEADER]: Becoming Leader. State: Replica: 083e444232ef4cc2a26fbee90b70b87d, State: Running, Role: LEADER
I20260619 02:53:36.213176 870 consensus_queue.cc:237] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb61eb911a24442acb8c3662c346f80" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 34647 } } peers { permanent_uuid: "011f109736ca46a0b59476e947fc20a3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43259 } } peers { permanent_uuid: "083e444232ef4cc2a26fbee90b70b87d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44933 } }
I20260619 02:53:36.218474 389 catalog_manager.cc:5495] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d reported cstate change: term changed from 0 to 1, leader changed from <none> to 083e444232ef4cc2a26fbee90b70b87d (127.18.134.3). New cstate: current_term: 1 leader_uuid: "083e444232ef4cc2a26fbee90b70b87d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb61eb911a24442acb8c3662c346f80" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 34647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "011f109736ca46a0b59476e947fc20a3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "083e444232ef4cc2a26fbee90b70b87d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44933 } health_report { overall_health: HEALTHY } } }
I20260619 02:53:36.242309 863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:36.236325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57562 (local address 127.18.134.3:44933)
0619 02:53:36.236494 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:36.236498 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:36.236515 (+ 17us) server_negotiation.cc:407] Connection header received
0619 02:53:36.236705 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:36.236709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:36.236798 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:36.236914 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:53:36.237349 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.238089 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.240131 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.240543 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.240632 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:36.241274 (+ 642us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:53:36.241767 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:53:36.241912 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:36.241992 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 02:53:36.242058 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 02:53:36.251533 668 raft_consensus.cc:1261] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 011f109736ca46a0b59476e947fc20a3 [term 1 FOLLOWER]: Refusing update from remote peer 083e444232ef4cc2a26fbee90b70b87d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:53:36.251535 534 raft_consensus.cc:1261] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 8eb61eb911a24442acb8c3662c346f80 [term 1 FOLLOWER]: Refusing update from remote peer 083e444232ef4cc2a26fbee90b70b87d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:53:36.252652 874 consensus_queue.cc:1035] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d [LEADER]: Connected to new peer: Peer: permanent_uuid: "011f109736ca46a0b59476e947fc20a3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:53:36.253006 870 consensus_queue.cc:1035] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb61eb911a24442acb8c3662c346f80" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 34647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 02:53:36.265728 847 tablet.cc:2369] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:53:36.269927 877 mvcc.cc:205] Tried to move back new op lower bound from 7298406876156006400 to 7298406876013965312. Current Snapshot: MvccSnapshot[applied={T|T < 7298406876156006400}]
I20260619 02:53:36.295984 389 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:53:36.296305 389 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:53:36.300588 389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91: Sending DeleteTablet for 3 replicas of tablet 5a5a01e8c36a4a558f96fe13fb3ae526
I20260619 02:53:36.301839 514 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a5a01e8c36a4a558f96fe13fb3ae526 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:53:36 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:36002
I20260619 02:53:36.302201 648 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a5a01e8c36a4a558f96fe13fb3ae526 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:53:36 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:56806
I20260619 02:53:36.302567 781 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a5a01e8c36a4a558f96fe13fb3ae526 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:53:36 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:57556
I20260619 02:53:36.303690 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 451
I20260619 02:53:36.321276 887 tablet_replica.cc:330] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 8eb61eb911a24442acb8c3662c346f80: stopping tablet replica
I20260619 02:53:36.321928 887 raft_consensus.cc:2229] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 8eb61eb911a24442acb8c3662c346f80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:53:36.326381 887 raft_consensus.cc:2258] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 8eb61eb911a24442acb8c3662c346f80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:53:36.327064 888 tablet_replica.cc:330] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 011f109736ca46a0b59476e947fc20a3: stopping tablet replica
I20260619 02:53:36.327811 888 raft_consensus.cc:2229] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 011f109736ca46a0b59476e947fc20a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:53:36.328475 888 raft_consensus.cc:2258] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 011f109736ca46a0b59476e947fc20a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:53:36.329788 889 tablet_replica.cc:330] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d: stopping tablet replica
I20260619 02:53:36.329981 887 ts_tablet_manager.cc:1905] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 8eb61eb911a24442acb8c3662c346f80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:53:36.330226 889 raft_consensus.cc:2229] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:53:36.331188 889 raft_consensus.cc:2258] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:53:36.332182 888 ts_tablet_manager.cc:1905] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 011f109736ca46a0b59476e947fc20a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:53:36.333237 889 ts_tablet_manager.cc:1905] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 19 02:53:36 dist-test-slave-9mr0 krb5kdc[345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.1@KRBTEST.COM
I20260619 02:53:36.340763 889 ts_tablet_manager.cc:1918] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:53:36.340967 889 log.cc:1196] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/wal/wals/5a5a01e8c36a4a558f96fe13fb3ae526
I20260619 02:53:36.341876 889 ts_tablet_manager.cc:1939] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 083e444232ef4cc2a26fbee90b70b87d: Deleting consensus metadata
I20260619 02:53:36.343858 376 catalog_manager.cc:4841] TS 083e444232ef4cc2a26fbee90b70b87d (127.18.134.3:44933): tablet 5a5a01e8c36a4a558f96fe13fb3ae526 (table test-table [id=973a6bdbf8d74f9095dff465e60e293a]) successfully deleted
I20260619 02:53:36.347915 887 ts_tablet_manager.cc:1918] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 8eb61eb911a24442acb8c3662c346f80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:53:36.348044 858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:36.304961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36008 (local address 127.18.134.1:34647)
0619 02:53:36.305151 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:36.305157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:36.328956 (+ 23799us) server_negotiation.cc:407] Connection header received
0619 02:53:36.329173 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:36.329179 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:36.329270 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:36.329378 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:36.331319 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.332401 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.334423 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.335044 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.335138 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:36.340567 (+ 5429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:36.340583 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:36.340603 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:36.340648 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:36.344688 (+ 4040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:36.345630 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:36.345640 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:36.345643 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:36.345826 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:36.346234 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:36.346239 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:36.346242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:36.346780 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:36.347100 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:36.347360 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 02:53:36.347606 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:53:36.348080 887 log.cc:1196] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 8eb61eb911a24442acb8c3662c346f80: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/wal/wals/5a5a01e8c36a4a558f96fe13fb3ae526
I20260619 02:53:36.348892 574 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:53:36.351178 888 ts_tablet_manager.cc:1918] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 011f109736ca46a0b59476e947fc20a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:53:36.351411 888 log.cc:1196] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 011f109736ca46a0b59476e947fc20a3: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/wal/wals/5a5a01e8c36a4a558f96fe13fb3ae526
I20260619 02:53:36.352139 888 ts_tablet_manager.cc:1939] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 011f109736ca46a0b59476e947fc20a3: Deleting consensus metadata
I20260619 02:53:36.354199 374 catalog_manager.cc:4841] TS 011f109736ca46a0b59476e947fc20a3 (127.18.134.2:43259): tablet 5a5a01e8c36a4a558f96fe13fb3ae526 (table test-table [id=973a6bdbf8d74f9095dff465e60e293a]) successfully deleted
I20260619 02:53:36.442011 887 ts_tablet_manager.cc:1939] T 5a5a01e8c36a4a558f96fe13fb3ae526 P 8eb61eb911a24442acb8c3662c346f80: Deleting consensus metadata
I20260619 02:53:36.443032 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 451
W20260619 02:53:36.464418 377 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.134.1:34647 (error 108)
I20260619 02:53:36.471484 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 585
I20260619 02:53:36.472246 859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:36.471714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:34647 (local address 127.0.0.1:36016)
0619 02:53:36.471951 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:36.472068 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:34647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
W20260619 02:53:36.472597 377 catalog_manager.cc:4584] TS 8eb61eb911a24442acb8c3662c346f80 (127.18.134.1:34647): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5a5a01e8c36a4a558f96fe13fb3ae526: Network error: Client connection negotiation failed: client connection to 127.18.134.1:34647: connect: Connection refused (error 111)
Jun 19 02:53:36 dist-test-slave-9mr0 krb5kdc[345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.2@KRBTEST.COM
I20260619 02:53:36.489233 862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:36.472672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56814 (local address 127.18.134.2:43259)
0619 02:53:36.473036 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:36.473043 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:36.473064 (+ 21us) server_negotiation.cc:407] Connection header received
0619 02:53:36.473129 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:36.473135 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:36.473217 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:36.473341 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:36.474613 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.475762 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.477198 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.477833 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.477962 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:36.482237 (+ 4275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:36.482257 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:36.482276 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:36.482327 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:36.486118 (+ 3791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:36.486931 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:36.486940 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:36.486944 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:36.487101 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:36.487489 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:36.487494 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:36.487497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:36.488002 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:36.488297 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:36.488601 (+ 304us) server_negotiation.cc:299] Negotiation successful
0619 02:53:36.488847 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238}
I20260619 02:53:36.489915 708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:53:36.526536 859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:36.525884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:34647 (local address 127.0.0.1:36026)
0619 02:53:36.526129 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:36.526311 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:34647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260619 02:53:36.527762 859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:36.527327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:34647 (local address 127.0.0.1:36028)
0619 02:53:36.527489 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:36.527579 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:34647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260619 02:53:36.583838 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 585
I20260619 02:53:36.588239 859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:36.587778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:34647 (local address 127.0.0.1:36032)
0619 02:53:36.587995 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:36.588089 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:34647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260619 02:53:36.589263 859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:36.588897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:34647 (local address 127.0.0.1:36038)
0619 02:53:36.589040 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:36.589124 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:34647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260619 02:53:36.603700 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 718
Jun 19 02:53:36 dist-test-slave-9mr0 krb5kdc[345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.3@KRBTEST.COM
I20260619 02:53:36.622076 863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:36.604511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57564 (local address 127.18.134.3:44933)
0619 02:53:36.604683 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:36.604687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:36.604762 (+ 75us) server_negotiation.cc:407] Connection header received
0619 02:53:36.604951 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:36.604955 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:36.605025 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:36.605124 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:36.606851 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.607786 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.609685 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:36.610195 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:36.610311 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:36.616090 (+ 5779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:36.616103 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:36.616121 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:36.616175 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:36.619190 (+ 3015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:36.620093 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:36.620100 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:36.620102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:36.620254 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:36.620674 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:36.620677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:36.620679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:36.621033 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:36.621260 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:36.621602 (+ 342us) server_negotiation.cc:299] Negotiation successful
0619 02:53:36.621769 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 02:53:36.622972 841 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:53:36.670095 859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:36.669531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:34647 (local address 127.0.0.1:36042)
0619 02:53:36.669788 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:36.669913 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:34647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260619 02:53:36.671236 859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:36.670801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:34647 (local address 127.0.0.1:36054)
0619 02:53:36.671005 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:36.671090 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:34647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260619 02:53:36.740716 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 718
I20260619 02:53:36.755465 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 358
I20260619 02:53:36.756596 420 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:53:36.865212 859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:36.864613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:34647 (local address 127.0.0.1:36068)
0619 02:53:36.864794 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:36.864896 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:34647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260619 02:53:36.865613 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 358
I20260619 02:53:36.883119 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:33207
--webserver_interface=127.18.134.62
--webserver_port=34703
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:33207
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:53:37.199463 896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:37.199745 896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:37.199841 896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:53:37.207124 896 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:53:37.207248 896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:37.207305 896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:37.207360 896 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:53:37.207403 896 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:53:37.217201 896 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:33207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:33207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=34703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:37.218423 896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:37.219707 896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:53:37.240109 905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:37.240298 902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:37.240294 903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:37.240814 896 server_base.cc:1034] running on GCE node
I20260619 02:53:37.241690 896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:37.243340 896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:37.244565 896 hybrid_clock.cc:648] HybridClock initialized: now 1781837617244500 us; error 49 us; skew 500 ppm
I20260619 02:53:37.244956 896 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:37 dist-test-slave-9mr0 krb5kdc[345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837617, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:37.248548 896 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:53:37.250337 896 webserver.cc:460] Webserver started at http://127.18.134.62:34703/ using document root <none> and password file <none>
I20260619 02:53:37.251176 896 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:37.251312 896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:53:37.257414 896 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 02:53:37.260406 912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:37.261790 896 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 02:53:37.261957 896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "6f92d2f13db7417da3983549b952dd91"
format_stamp: "Formatted at 2026-06-19 02:53:34 on dist-test-slave-9mr0"
server_key: "a666cd18c1b6770c6db96d1f4147971d"
server_key_iv: "8fa7a41f11bd84b00563b31988a41740"
server_key_version: "encryptionkey@0"
I20260619 02:53:37.263132 896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:37.281056 896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:37.285465 896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:37.285748 896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:37.303881 896 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:33207
I20260619 02:53:37.303911 963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:33207 every 8 connection(s)
I20260619 02:53:37.305440 896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:53:37.312534 964 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:53:37.313527 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 896
I20260619 02:53:37.314620 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:34647
--local_ip_for_outbound_sockets=127.18.134.1
--tserver_master_addrs=127.18.134.62:33207
--webserver_port=36467
--webserver_interface=127.18.134.1
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:53:37.316761 964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91: Bootstrap starting.
I20260619 02:53:37.336226 964 log.cc:824] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91: Log is configured to *not* fsync() on all Append() calls
I20260619 02:53:37.348421 964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:53:37.348919 964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91: Bootstrap complete.
I20260619 02:53:37.358076 964 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f92d2f13db7417da3983549b952dd91" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33207 } }
I20260619 02:53:37.358986 964 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f92d2f13db7417da3983549b952dd91, State: Initialized, Role: FOLLOWER
I20260619 02:53:37.359588 964 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f92d2f13db7417da3983549b952dd91" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33207 } }
I20260619 02:53:37.359817 964 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:53:37.359931 964 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:53:37.360085 964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:53:37.363941 964 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f92d2f13db7417da3983549b952dd91" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33207 } }
I20260619 02:53:37.364470 964 leader_election.cc:302] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f92d2f13db7417da3983549b952dd91; no voters:
I20260619 02:53:37.365059 964 leader_election.cc:288] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 02:53:37.365206 968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 02:53:37.366076 968 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [term 2 LEADER]: Becoming Leader. State: Replica: 6f92d2f13db7417da3983549b952dd91, State: Running, Role: LEADER
I20260619 02:53:37.366658 968 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f92d2f13db7417da3983549b952dd91" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33207 } }
I20260619 02:53:37.367405 964 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:53:37.369284 970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f92d2f13db7417da3983549b952dd91. Latest consensus state: current_term: 2 leader_uuid: "6f92d2f13db7417da3983549b952dd91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f92d2f13db7417da3983549b952dd91" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33207 } } }
I20260619 02:53:37.369388 969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6f92d2f13db7417da3983549b952dd91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f92d2f13db7417da3983549b952dd91" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33207 } } }
I20260619 02:53:37.369653 970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [sys.catalog]: This master's current role is: LEADER
I20260619 02:53:37.369751 969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [sys.catalog]: This master's current role is: LEADER
I20260619 02:53:37.371044 973 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:53:37.379572 973 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:53:37.381088 973 catalog_manager.cc:1259] Loaded cluster ID: c8aa49fb650f439e845d95a09df7e6f9
I20260619 02:53:37.381206 973 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:53:37.384449 973 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260619 02:53:37.403475 973 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260619 02:53:37.423301 973 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:53:37.425079 973 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91: Loaded TSK: 0
I20260619 02:53:37.426661 973 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 02:53:37.656800 966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:37.657100 966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:37.657240 966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:53:37.664961 966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:37.665091 966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:37.665249 966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:53:37.675374 966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:34647
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=36467
--tserver_master_addrs=127.18.134.62:33207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:37.676659 966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:37.677966 966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:53:37.692879 992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:37.692905 994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:37.692957 991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:37.694368 966 server_base.cc:1034] running on GCE node
I20260619 02:53:37.695192 966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:37.696463 966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:37.697736 966 hybrid_clock.cc:648] HybridClock initialized: now 1781837617697660 us; error 49 us; skew 500 ppm
I20260619 02:53:37.698175 966 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:37 dist-test-slave-9mr0 krb5kdc[345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837617, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:37.702142 966 init.cc:377] Logged in from keytab as kudu/127.18.134.1@KRBTEST.COM (short username kudu)
I20260619 02:53:37.703967 966 webserver.cc:460] Webserver started at http://127.18.134.1:36467/ using document root <none> and password file <none>
I20260619 02:53:37.704890 966 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:37.705044 966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:53:37.711939 966 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 02:53:37.715102 1001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:37.716753 966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 02:53:37.716921 966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "8eb61eb911a24442acb8c3662c346f80"
format_stamp: "Formatted at 2026-06-19 02:53:34 on dist-test-slave-9mr0"
server_key: "650df52c8f80a9b31296004d51d0b22a"
server_key_iv: "2632c9cb4bc69fab6feab44663ff1d34"
server_key_version: "encryptionkey@0"
I20260619 02:53:37.718227 966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:37.740418 966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:37.745658 966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:37.745942 966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:37.747145 966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:53:37.748729 966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:53:37.748842 966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:37.748943 966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:53:37.749014 966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:37.780239 966 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:34647
I20260619 02:53:37.780315 1113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:34647 every 8 connection(s)
I20260619 02:53:37.781636 966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:53:37.787104 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 966
I20260619 02:53:37.787835 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:43259
--local_ip_for_outbound_sockets=127.18.134.2
--tserver_master_addrs=127.18.134.62:33207
--webserver_port=42729
--webserver_interface=127.18.134.2
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:53:37 dist-test-slave-9mr0 krb5kdc[345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837617, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:53:37.805546 1117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:37.784521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.1:55695)
0619 02:53:37.785256 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:37.785306 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:37.786962 (+ 1656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:37.788528 (+ 1566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:37.788535 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:37.789032 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:53:37.789601 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:37.789620 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:37.791742 (+ 2122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:37.791747 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:37.793047 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:37.793061 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:37.794399 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:37.794407 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:37.794712 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:37.795499 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:53:37.795530 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:53:37.797759 (+ 2229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:53:37.801132 (+ 3373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:37.801142 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:37.801159 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:37.801596 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:37.802050 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:37.802056 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:37.802060 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:37.802255 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:37.803069 (+ 814us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:53:37.803077 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:53:37.803415 (+ 338us) client_negotiation.cc:769] Sending connection context
0619 02:53:37.803655 (+ 240us) client_negotiation.cc:240] Negotiation successful
0619 02:53:37.804236 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":300,"threads_started":1}
I20260619 02:53:37.805546 1116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:37.784398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:55695 (local address 127.18.134.62:33207)
0619 02:53:37.785037 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:37.785053 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:37.785424 (+ 371us) server_negotiation.cc:407] Connection header received
0619 02:53:37.787536 (+ 2112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:37.787554 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:37.788033 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:37.788405 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:37.789824 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:37.791564 (+ 1740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:37.793195 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:37.794160 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:37.794346 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:37.798064 (+ 3718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:37.798073 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:37.798089 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:37.798133 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:37.800971 (+ 2838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:37.801738 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:37.801746 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:37.801751 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:37.801929 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:37.802372 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:37.802376 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:37.802378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:37.802932 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:37.803118 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:37.803902 (+ 784us) server_negotiation.cc:299] Negotiation successful
0619 02:53:37.804295 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":242,"threads_started":1}
I20260619 02:53:37.808894 1114 heartbeater.cc:347] Connected to a master server at 127.18.134.62:33207
I20260619 02:53:37.809185 1114 heartbeater.cc:464] Registering TS with master...
I20260619 02:53:37.809990 1114 heartbeater.cc:511] Master 127.18.134.62:33207 requested a full tablet report, sending...
I20260619 02:53:37.812376 929 ts_manager.cc:194] Registered new tserver with Master: 8eb61eb911a24442acb8c3662c346f80 (127.18.134.1:34647)
I20260619 02:53:37.814474 929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.1@KRBTEST.COM'} at 127.18.134.1:55695
W20260619 02:53:38.107507 1118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:38.107784 1118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:38.107887 1118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:53:38.115216 1118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:38.115329 1118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:38.115439 1118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:53:38.125257 1118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:43259
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=42729
--tserver_master_addrs=127.18.134.62:33207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:38.126470 1118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:38.127737 1118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:53:38.140826 1126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:38.140878 1124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:38.140781 1123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:38.141232 1118 server_base.cc:1034] running on GCE node
I20260619 02:53:38.142118 1118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:38.143131 1118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:38.144363 1118 hybrid_clock.cc:648] HybridClock initialized: now 1781837618144320 us; error 33 us; skew 500 ppm
I20260619 02:53:38.144692 1118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:38 dist-test-slave-9mr0 krb5kdc[345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837618, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:38.147799 1118 init.cc:377] Logged in from keytab as kudu/127.18.134.2@KRBTEST.COM (short username kudu)
I20260619 02:53:38.149251 1118 webserver.cc:460] Webserver started at http://127.18.134.2:42729/ using document root <none> and password file <none>
I20260619 02:53:38.149912 1118 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:38.150043 1118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:53:38.155437 1118 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260619 02:53:38.158114 1133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:38.159565 1118 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 02:53:38.159739 1118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "011f109736ca46a0b59476e947fc20a3"
format_stamp: "Formatted at 2026-06-19 02:53:35 on dist-test-slave-9mr0"
server_key: "794a5cd224ed315bd923c3e7fdcde710"
server_key_iv: "76c8d3f264a2dc32f478810894caf48e"
server_key_version: "encryptionkey@0"
I20260619 02:53:38.160696 1118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:38.173676 1118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:38.177599 1118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:38.177875 1118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:38.178874 1118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:53:38.180285 1118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:53:38.180382 1118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:38.180495 1118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:53:38.180562 1118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:38.210529 1118 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:43259
I20260619 02:53:38.210583 1245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:43259 every 8 connection(s)
I20260619 02:53:38.211987 1118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:53:38.220793 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 1118
I20260619 02:53:38.221405 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:44933
--local_ip_for_outbound_sockets=127.18.134.3
--tserver_master_addrs=127.18.134.62:33207
--webserver_port=43619
--webserver_interface=127.18.134.3
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:53:38 dist-test-slave-9mr0 krb5kdc[345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837618, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:53:38.232427 1116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:38.214695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:52041 (local address 127.18.134.62:33207)
0619 02:53:38.214893 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:38.214898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:38.216004 (+ 1106us) server_negotiation.cc:407] Connection header received
0619 02:53:38.217832 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:38.217838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:38.217940 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:38.218030 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:38.219646 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.220366 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.221981 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.222398 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.222507 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:38.226147 (+ 3640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:38.226152 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:38.226155 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:38.226204 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:38.229135 (+ 2931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:38.230004 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:38.230009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:38.230012 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:38.230158 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:38.230685 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:38.230690 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:38.230693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:38.231031 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:38.231165 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:38.231872 (+ 707us) server_negotiation.cc:299] Negotiation successful
0619 02:53:38.232104 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 02:53:38.233294 1248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:38.215150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.2:52041)
0619 02:53:38.215862 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:38.215898 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:38.217542 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:38.218293 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:38.218302 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:38.218892 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:53:38.219470 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:38.219489 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.220521 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.220525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:38.221840 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:38.221853 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.222614 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.222658 (+ 44us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:38.222864 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:38.223511 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:53:38.223533 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:53:38.225887 (+ 2354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:53:38.229291 (+ 3404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:38.229298 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:38.229312 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:38.229866 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:38.230339 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:38.230342 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:38.230345 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:38.230546 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:38.231186 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:53:38.231192 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:53:38.231517 (+ 325us) client_negotiation.cc:769] Sending connection context
0619 02:53:38.231745 (+ 228us) client_negotiation.cc:240] Negotiation successful
0619 02:53:38.232192 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":263,"threads_started":1}
I20260619 02:53:38.234804 1246 heartbeater.cc:347] Connected to a master server at 127.18.134.62:33207
I20260619 02:53:38.235126 1246 heartbeater.cc:464] Registering TS with master...
I20260619 02:53:38.236028 1246 heartbeater.cc:511] Master 127.18.134.62:33207 requested a full tablet report, sending...
I20260619 02:53:38.237761 929 ts_manager.cc:194] Registered new tserver with Master: 011f109736ca46a0b59476e947fc20a3 (127.18.134.2:43259)
I20260619 02:53:38.238706 929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.2@KRBTEST.COM'} at 127.18.134.2:52041
W20260619 02:53:38.538228 1249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:38.538503 1249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:38.538666 1249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:53:38.545977 1249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:38.546087 1249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:38.546232 1249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:53:38.556089 1249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:44933
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=43619
--tserver_master_addrs=127.18.134.62:33207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:38.557366 1249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:38.558713 1249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:53:38.571727 1254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:38.571651 1257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:38.571702 1255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:38.572570 1249 server_base.cc:1034] running on GCE node
I20260619 02:53:38.573217 1249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:38.574188 1249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:38.575407 1249 hybrid_clock.cc:648] HybridClock initialized: now 1781837618575349 us; error 49 us; skew 500 ppm
I20260619 02:53:38.575745 1249 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:38 dist-test-slave-9mr0 krb5kdc[345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837618, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:38.579001 1249 init.cc:377] Logged in from keytab as kudu/127.18.134.3@KRBTEST.COM (short username kudu)
I20260619 02:53:38.580497 1249 webserver.cc:460] Webserver started at http://127.18.134.3:43619/ using document root <none> and password file <none>
I20260619 02:53:38.581158 1249 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:38.581279 1249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:53:38.586741 1249 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:53:38.589377 1264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:38.590706 1249 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:53:38.590844 1249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "083e444232ef4cc2a26fbee90b70b87d"
format_stamp: "Formatted at 2026-06-19 02:53:35 on dist-test-slave-9mr0"
server_key: "582892f9e0421d7d602e1303a4c62d90"
server_key_iv: "194c56670a55ce2eaf708d1617bac467"
server_key_version: "encryptionkey@0"
I20260619 02:53:38.591957 1249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:38.616587 1249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:38.621126 1249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:38.621373 1249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:38.622364 1249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:53:38.623739 1249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:53:38.623826 1249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:38.623900 1249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:53:38.623951 1249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:38.654294 1249 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:44933
I20260619 02:53:38.654371 1376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:44933 every 8 connection(s)
I20260619 02:53:38.655759 1249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:53:38.665108 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 1249
Jun 19 02:53:38 dist-test-slave-9mr0 krb5kdc[345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837618, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:53:38.676949 1116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:38.658403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:55117 (local address 127.18.134.62:33207)
0619 02:53:38.658624 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:38.658661 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:38.659858 (+ 1197us) server_negotiation.cc:407] Connection header received
0619 02:53:38.661641 (+ 1783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:38.661648 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:38.661734 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:38.661826 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:38.663430 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.664158 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.666106 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.666940 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.667079 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:38.670868 (+ 3789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:38.670873 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:38.670876 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:38.670906 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:38.673699 (+ 2793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:38.674473 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:38.674477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:38.674479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:38.674606 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:38.675118 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:38.675121 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:38.675123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:38.675396 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:38.675503 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:38.676434 (+ 931us) server_negotiation.cc:299] Negotiation successful
0619 02:53:38.676672 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260619 02:53:38.678018 1379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:38.658930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.3:55117)
0619 02:53:38.659674 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:38.659713 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:38.661371 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:38.662064 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:38.662072 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:38.662569 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:53:38.663263 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:38.663284 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.664312 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.664315 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:38.665954 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:38.665969 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.667143 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.667149 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:38.667398 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:38.668070 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:53:38.668092 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:53:38.670653 (+ 2561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:53:38.673834 (+ 3181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:38.673844 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:38.673862 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:38.674352 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:38.674730 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:38.674736 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:38.674739 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:38.675010 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:38.675516 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:53:38.675524 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:53:38.675987 (+ 463us) client_negotiation.cc:769] Sending connection context
0619 02:53:38.676272 (+ 285us) client_negotiation.cc:240] Negotiation successful
0619 02:53:38.676664 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":293,"threads_started":1}
I20260619 02:53:38.679541 1377 heartbeater.cc:347] Connected to a master server at 127.18.134.62:33207
I20260619 02:53:38.679852 1377 heartbeater.cc:464] Registering TS with master...
I20260619 02:53:38.680536 1377 heartbeater.cc:511] Master 127.18.134.62:33207 requested a full tablet report, sending...
I20260619 02:53:38.681943 929 ts_manager.cc:194] Registered new tserver with Master: 083e444232ef4cc2a26fbee90b70b87d (127.18.134.3:44933)
I20260619 02:53:38.683040 1380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:38.666079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37182 (local address 127.18.134.62:33207)
0619 02:53:38.666569 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:38.666575 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:38.667752 (+ 1177us) server_negotiation.cc:407] Connection header received
0619 02:53:38.668175 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:38.668179 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:38.668301 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:38.668407 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:38.669894 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.670950 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.672466 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.673156 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.673292 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:38.676106 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:38.676110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:38.676114 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:38.676141 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:38.678959 (+ 2818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:38.681199 (+ 2240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:38.681203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:38.681205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:38.681325 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:38.681687 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:38.681690 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:38.681692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:38.681978 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:38.682206 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:38.682526 (+ 320us) server_negotiation.cc:299] Negotiation successful
0619 02:53:38.682736 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":241,"threads_started":1}
I20260619 02:53:38.683108 929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.3@KRBTEST.COM'} at 127.18.134.3:55117
I20260619 02:53:38.685374 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:53:38.691581 1380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:38.686220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37190 (local address 127.18.134.62:33207)
0619 02:53:38.686389 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:38.686393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:38.686834 (+ 441us) server_negotiation.cc:407] Connection header received
0619 02:53:38.686971 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:38.686974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:38.687101 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:38.687216 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:53:38.687597 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.688253 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.689749 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.690137 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.690217 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:38.690623 (+ 406us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:53:38.691142 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:53:38.691243 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:38.691336 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 02:53:38.691399 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 02:53:38.692405 928 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:37190:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:53:38.699721 928 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:53:38.742498 1388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:38.711340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36080 (local address 127.18.134.1:34647)
0619 02:53:38.726659 (+ 15319us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:38.726669 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:38.731859 (+ 5190us) server_negotiation.cc:407] Connection header received
0619 02:53:38.731981 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:38.731990 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:38.732285 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:38.732469 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:53:38.733525 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.736464 (+ 2939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.739695 (+ 3231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.741217 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.741372 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:38.741584 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:38.741993 (+ 409us) server_negotiation.cc:299] Negotiation successful
0619 02:53:38.742164 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15096,"thread_start_us":15324,"threads_started":1}
I20260619 02:53:38.742544 1390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:38.711170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:34647 (local address 127.0.0.1:36080)
0619 02:53:38.731396 (+ 20226us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:38.731443 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:38.731624 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:38.732767 (+ 1143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:38.732788 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:38.732830 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:53:38.733309 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:38.733322 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.736642 (+ 3320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.736648 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:38.739354 (+ 2706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:38.739372 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.741487 (+ 2115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.741494 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:38.741728 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:38.741789 (+ 61us) client_negotiation.cc:769] Sending connection context
0619 02:53:38.741924 (+ 135us) client_negotiation.cc:240] Negotiation successful
0619 02:53:38.742116 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19973,"thread_start_us":28800,"threads_started":1}
I20260619 02:53:38.746522 1049 tablet_service.cc:1467] Processing CreateTablet for tablet 593c1ec8a0f7461da2f4fa79be5973dd (DEFAULT_TABLE table=test-table [id=c78d4053df434d1d86f5a3931d347746]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:53:38.747563 1389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:38.711514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:44933 (local address 127.0.0.1:57568)
0619 02:53:38.739960 (+ 28446us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:38.739984 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:38.740232 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:38.740870 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:38.740874 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:38.740899 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:53:38.741228 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:38.741241 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.743339 (+ 2098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.743344 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:38.745049 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:38.745063 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.746756 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.746763 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:38.746953 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:38.746987 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 02:53:38.747084 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 02:53:38.747213 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28312,"thread_start_us":28171,"threads_started":1}
I20260619 02:53:38.747637 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:38.712351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57568 (local address 127.18.134.3:44933)
0619 02:53:38.739287 (+ 26936us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:38.739297 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:38.740104 (+ 807us) server_negotiation.cc:407] Connection header received
0619 02:53:38.740457 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:38.740464 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:38.740681 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:38.740854 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:53:38.741442 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.743203 (+ 1761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.745224 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.746529 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.746695 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:38.746909 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:38.747135 (+ 226us) server_negotiation.cc:299] Negotiation successful
0619 02:53:38.747302 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26739,"thread_start_us":26944,"threads_started":1}
I20260619 02:53:38.748283 1049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 593c1ec8a0f7461da2f4fa79be5973dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:53:38.748411 1387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:38.712003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:43259 (local address 127.0.0.1:56818)
0619 02:53:38.740223 (+ 28220us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:38.740239 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:38.740348 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:38.741135 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:38.741140 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:38.741175 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:53:38.741639 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:38.741651 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.744355 (+ 2704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.744360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:38.745876 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:38.745887 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.747616 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.747743 (+ 127us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:38.747867 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:38.747890 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 02:53:38.747969 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 02:53:38.748090 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28109,"thread_start_us":18989,"threads_started":1}
I20260619 02:53:38.748457 1392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:38.712939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56818 (local address 127.18.134.2:43259)
0619 02:53:38.738488 (+ 25549us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:38.738498 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:38.740363 (+ 1865us) server_negotiation.cc:407] Connection header received
0619 02:53:38.740490 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:38.740497 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:38.740733 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:38.740919 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:53:38.743035 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.744220 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.746016 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.747086 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.747205 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:38.747464 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:38.748029 (+ 565us) server_negotiation.cc:299] Negotiation successful
0619 02:53:38.748193 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25357,"thread_start_us":25574,"threads_started":1}
I20260619 02:53:38.752146 1181 tablet_service.cc:1467] Processing CreateTablet for tablet 593c1ec8a0f7461da2f4fa79be5973dd (DEFAULT_TABLE table=test-table [id=c78d4053df434d1d86f5a3931d347746]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:53:38.752722 1312 tablet_service.cc:1467] Processing CreateTablet for tablet 593c1ec8a0f7461da2f4fa79be5973dd (DEFAULT_TABLE table=test-table [id=c78d4053df434d1d86f5a3931d347746]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:53:38.753556 1181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 593c1ec8a0f7461da2f4fa79be5973dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:53:38.753796 1312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 593c1ec8a0f7461da2f4fa79be5973dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:53:38.763753 1393 tablet_bootstrap.cc:492] T 593c1ec8a0f7461da2f4fa79be5973dd P 8eb61eb911a24442acb8c3662c346f80: Bootstrap starting.
I20260619 02:53:38.766777 1394 tablet_bootstrap.cc:492] T 593c1ec8a0f7461da2f4fa79be5973dd P 083e444232ef4cc2a26fbee90b70b87d: Bootstrap starting.
I20260619 02:53:38.768476 1395 tablet_bootstrap.cc:492] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3: Bootstrap starting.
I20260619 02:53:38.769409 1393 tablet_bootstrap.cc:654] T 593c1ec8a0f7461da2f4fa79be5973dd P 8eb61eb911a24442acb8c3662c346f80: Neither blocks nor log segments found. Creating new log.
I20260619 02:53:38.770052 1394 tablet_bootstrap.cc:654] T 593c1ec8a0f7461da2f4fa79be5973dd P 083e444232ef4cc2a26fbee90b70b87d: Neither blocks nor log segments found. Creating new log.
I20260619 02:53:38.770711 1393 log.cc:824] T 593c1ec8a0f7461da2f4fa79be5973dd P 8eb61eb911a24442acb8c3662c346f80: Log is configured to *not* fsync() on all Append() calls
I20260619 02:53:38.771023 1394 log.cc:824] T 593c1ec8a0f7461da2f4fa79be5973dd P 083e444232ef4cc2a26fbee90b70b87d: Log is configured to *not* fsync() on all Append() calls
I20260619 02:53:38.771908 1395 tablet_bootstrap.cc:654] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3: Neither blocks nor log segments found. Creating new log.
I20260619 02:53:38.773039 1395 log.cc:824] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3: Log is configured to *not* fsync() on all Append() calls
I20260619 02:53:38.773851 1393 tablet_bootstrap.cc:492] T 593c1ec8a0f7461da2f4fa79be5973dd P 8eb61eb911a24442acb8c3662c346f80: No bootstrap required, opened a new log
I20260619 02:53:38.774098 1393 ts_tablet_manager.cc:1397] T 593c1ec8a0f7461da2f4fa79be5973dd P 8eb61eb911a24442acb8c3662c346f80: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260619 02:53:38.775008 1394 tablet_bootstrap.cc:492] T 593c1ec8a0f7461da2f4fa79be5973dd P 083e444232ef4cc2a26fbee90b70b87d: No bootstrap required, opened a new log
I20260619 02:53:38.775256 1394 ts_tablet_manager.cc:1397] T 593c1ec8a0f7461da2f4fa79be5973dd P 083e444232ef4cc2a26fbee90b70b87d: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260619 02:53:38.775856 1395 tablet_bootstrap.cc:492] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3: No bootstrap required, opened a new log
I20260619 02:53:38.776098 1395 ts_tablet_manager.cc:1397] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260619 02:53:38.784175 1394 raft_consensus.cc:348] T 593c1ec8a0f7461da2f4fa79be5973dd P 083e444232ef4cc2a26fbee90b70b87d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb61eb911a24442acb8c3662c346f80" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 34647 } } peers { permanent_uuid: "083e444232ef4cc2a26fbee90b70b87d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44933 } } peers { permanent_uuid: "011f109736ca46a0b59476e947fc20a3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43259 } }
I20260619 02:53:38.784673 1394 raft_consensus.cc:374] T 593c1ec8a0f7461da2f4fa79be5973dd P 083e444232ef4cc2a26fbee90b70b87d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:53:38.784773 1394 raft_consensus.cc:729] T 593c1ec8a0f7461da2f4fa79be5973dd P 083e444232ef4cc2a26fbee90b70b87d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 083e444232ef4cc2a26fbee90b70b87d, State: Initialized, Role: FOLLOWER
I20260619 02:53:38.784833 1395 raft_consensus.cc:348] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb61eb911a24442acb8c3662c346f80" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 34647 } } peers { permanent_uuid: "083e444232ef4cc2a26fbee90b70b87d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44933 } } peers { permanent_uuid: "011f109736ca46a0b59476e947fc20a3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43259 } }
I20260619 02:53:38.785264 1395 raft_consensus.cc:374] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:53:38.785362 1395 raft_consensus.cc:729] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 011f109736ca46a0b59476e947fc20a3, State: Initialized, Role: FOLLOWER
I20260619 02:53:38.785418 1393 raft_consensus.cc:348] T 593c1ec8a0f7461da2f4fa79be5973dd P 8eb61eb911a24442acb8c3662c346f80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb61eb911a24442acb8c3662c346f80" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 34647 } } peers { permanent_uuid: "083e444232ef4cc2a26fbee90b70b87d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44933 } } peers { permanent_uuid: "011f109736ca46a0b59476e947fc20a3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43259 } }
I20260619 02:53:38.785547 1394 consensus_queue.cc:260] T 593c1ec8a0f7461da2f4fa79be5973dd P 083e444232ef4cc2a26fbee90b70b87d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb61eb911a24442acb8c3662c346f80" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 34647 } } peers { permanent_uuid: "083e444232ef4cc2a26fbee90b70b87d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44933 } } peers { permanent_uuid: "011f109736ca46a0b59476e947fc20a3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43259 } }
I20260619 02:53:38.785841 1393 raft_consensus.cc:374] T 593c1ec8a0f7461da2f4fa79be5973dd P 8eb61eb911a24442acb8c3662c346f80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:53:38.785936 1393 raft_consensus.cc:729] T 593c1ec8a0f7461da2f4fa79be5973dd P 8eb61eb911a24442acb8c3662c346f80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eb61eb911a24442acb8c3662c346f80, State: Initialized, Role: FOLLOWER
I20260619 02:53:38.786019 1395 consensus_queue.cc:260] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb61eb911a24442acb8c3662c346f80" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 34647 } } peers { permanent_uuid: "083e444232ef4cc2a26fbee90b70b87d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44933 } } peers { permanent_uuid: "011f109736ca46a0b59476e947fc20a3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43259 } }
I20260619 02:53:38.786541 1393 consensus_queue.cc:260] T 593c1ec8a0f7461da2f4fa79be5973dd P 8eb61eb911a24442acb8c3662c346f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb61eb911a24442acb8c3662c346f80" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 34647 } } peers { permanent_uuid: "083e444232ef4cc2a26fbee90b70b87d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44933 } } peers { permanent_uuid: "011f109736ca46a0b59476e947fc20a3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43259 } }
I20260619 02:53:38.794884 1114 heartbeater.cc:503] Master 127.18.134.62:33207 was elected leader, sending a full tablet report...
I20260619 02:53:38.796867 1393 ts_tablet_manager.cc:1428] T 593c1ec8a0f7461da2f4fa79be5973dd P 8eb61eb911a24442acb8c3662c346f80: Time spent starting tablet: real 0.023s user 0.016s sys 0.005s
I20260619 02:53:38.809150 1400 raft_consensus.cc:482] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:53:38.809239 1246 heartbeater.cc:503] Master 127.18.134.62:33207 was elected leader, sending a full tablet report...
I20260619 02:53:38.810150 1377 heartbeater.cc:503] Master 127.18.134.62:33207 was elected leader, sending a full tablet report...
I20260619 02:53:38.810050 1395 ts_tablet_manager.cc:1428] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3: Time spent starting tablet: real 0.034s user 0.023s sys 0.008s
I20260619 02:53:38.810361 1400 raft_consensus.cc:504] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb61eb911a24442acb8c3662c346f80" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 34647 } } peers { permanent_uuid: "083e444232ef4cc2a26fbee90b70b87d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44933 } } peers { permanent_uuid: "011f109736ca46a0b59476e947fc20a3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43259 } }
I20260619 02:53:38.811542 1394 ts_tablet_manager.cc:1428] T 593c1ec8a0f7461da2f4fa79be5973dd P 083e444232ef4cc2a26fbee90b70b87d: Time spent starting tablet: real 0.036s user 0.019s sys 0.015s
I20260619 02:53:38.813469 1400 leader_election.cc:288] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eb61eb911a24442acb8c3662c346f80 (127.18.134.1:34647), 083e444232ef4cc2a26fbee90b70b87d (127.18.134.3:44933)
I20260619 02:53:38.819525 1404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:38.812904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:34647 (local address 127.18.134.2:32985)
0619 02:53:38.813990 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:38.814002 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:38.814109 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:38.814511 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:38.814516 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:38.814540 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:53:38.814923 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:38.814933 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.816124 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.816130 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:38.817658 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:38.817670 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.818959 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.818965 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:38.819091 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:38.819114 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 02:53:38.819183 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 02:53:38.819278 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":990,"thread_start_us":893,"threads_started":1}
I20260619 02:53:38.819617 1388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:38.812677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:32985 (local address 127.18.134.1:34647)
0619 02:53:38.812878 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:38.812885 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:38.814089 (+ 1204us) server_negotiation.cc:407] Connection header received
0619 02:53:38.814214 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:38.814219 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:38.814289 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:38.814368 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:53:38.815070 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.816004 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.817860 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.818750 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.818868 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:38.818923 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:38.819228 (+ 305us) server_negotiation.cc:299] Negotiation successful
0619 02:53:38.819367 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 02:53:38.820541 1069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "593c1ec8a0f7461da2f4fa79be5973dd" candidate_uuid: "011f109736ca46a0b59476e947fc20a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb61eb911a24442acb8c3662c346f80" is_pre_election: true
I20260619 02:53:38.820828 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:38.813627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:37461 (local address 127.18.134.3:44933)
0619 02:53:38.813810 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:38.813816 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:38.813834 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:53:38.813900 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:38.813904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:38.813974 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:38.814058 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:53:38.815391 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.816373 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.819020 (+ 2647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.820065 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.820181 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:38.820231 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:38.820571 (+ 340us) server_negotiation.cc:299] Negotiation successful
0619 02:53:38.820657 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 02:53:38.820983 1403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:38.812624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:44933 (local address 127.18.134.2:37461)
0619 02:53:38.813505 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:38.813525 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:38.813654 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:38.814102 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:38.814106 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:38.814128 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:53:38.815176 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:38.815189 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.817190 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.817195 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:38.818860 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:38.818874 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.820253 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.820259 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:38.820431 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:38.820462 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 02:53:38.820558 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 02:53:38.820682 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"thread_start_us":624,"threads_started":1}
I20260619 02:53:38.821141 1069 raft_consensus.cc:2454] T 593c1ec8a0f7461da2f4fa79be5973dd P 8eb61eb911a24442acb8c3662c346f80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 011f109736ca46a0b59476e947fc20a3 in term 0.
I20260619 02:53:38.821759 1137 leader_election.cc:302] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 011f109736ca46a0b59476e947fc20a3, 8eb61eb911a24442acb8c3662c346f80; no voters:
I20260619 02:53:38.821764 1332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "593c1ec8a0f7461da2f4fa79be5973dd" candidate_uuid: "011f109736ca46a0b59476e947fc20a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "083e444232ef4cc2a26fbee90b70b87d" is_pre_election: true
I20260619 02:53:38.822094 1400 raft_consensus.cc:2781] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:53:38.822218 1400 raft_consensus.cc:482] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:53:38.822194 1332 raft_consensus.cc:2454] T 593c1ec8a0f7461da2f4fa79be5973dd P 083e444232ef4cc2a26fbee90b70b87d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 011f109736ca46a0b59476e947fc20a3 in term 0.
I20260619 02:53:38.822314 1400 raft_consensus.cc:3037] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:53:38.824638 1400 raft_consensus.cc:504] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb61eb911a24442acb8c3662c346f80" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 34647 } } peers { permanent_uuid: "083e444232ef4cc2a26fbee90b70b87d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44933 } } peers { permanent_uuid: "011f109736ca46a0b59476e947fc20a3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43259 } }
I20260619 02:53:38.825222 1400 leader_election.cc:288] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3 [CANDIDATE]: Term 1 election: Requested vote from peers 8eb61eb911a24442acb8c3662c346f80 (127.18.134.1:34647), 083e444232ef4cc2a26fbee90b70b87d (127.18.134.3:44933)
I20260619 02:53:38.825636 1332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "593c1ec8a0f7461da2f4fa79be5973dd" candidate_uuid: "011f109736ca46a0b59476e947fc20a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "083e444232ef4cc2a26fbee90b70b87d"
I20260619 02:53:38.825646 1069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "593c1ec8a0f7461da2f4fa79be5973dd" candidate_uuid: "011f109736ca46a0b59476e947fc20a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eb61eb911a24442acb8c3662c346f80"
I20260619 02:53:38.825804 1332 raft_consensus.cc:3037] T 593c1ec8a0f7461da2f4fa79be5973dd P 083e444232ef4cc2a26fbee90b70b87d [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:53:38.825824 1069 raft_consensus.cc:3037] T 593c1ec8a0f7461da2f4fa79be5973dd P 8eb61eb911a24442acb8c3662c346f80 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:53:38.827960 1332 raft_consensus.cc:2454] T 593c1ec8a0f7461da2f4fa79be5973dd P 083e444232ef4cc2a26fbee90b70b87d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 011f109736ca46a0b59476e947fc20a3 in term 1.
I20260619 02:53:38.827986 1069 raft_consensus.cc:2454] T 593c1ec8a0f7461da2f4fa79be5973dd P 8eb61eb911a24442acb8c3662c346f80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 011f109736ca46a0b59476e947fc20a3 in term 1.
I20260619 02:53:38.828326 1136 leader_election.cc:302] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 011f109736ca46a0b59476e947fc20a3, 083e444232ef4cc2a26fbee90b70b87d; no voters:
I20260619 02:53:38.828596 1400 raft_consensus.cc:2781] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:53:38.828732 1400 raft_consensus.cc:686] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3 [term 1 LEADER]: Becoming Leader. State: Replica: 011f109736ca46a0b59476e947fc20a3, State: Running, Role: LEADER
I20260619 02:53:38.829174 1400 consensus_queue.cc:237] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb61eb911a24442acb8c3662c346f80" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 34647 } } peers { permanent_uuid: "083e444232ef4cc2a26fbee90b70b87d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44933 } } peers { permanent_uuid: "011f109736ca46a0b59476e947fc20a3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43259 } }
I20260619 02:53:38.833755 928 catalog_manager.cc:5495] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 011f109736ca46a0b59476e947fc20a3 (127.18.134.2). New cstate: current_term: 1 leader_uuid: "011f109736ca46a0b59476e947fc20a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eb61eb911a24442acb8c3662c346f80" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 34647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "083e444232ef4cc2a26fbee90b70b87d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "011f109736ca46a0b59476e947fc20a3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 43259 } health_report { overall_health: HEALTHY } } }
I20260619 02:53:38.852998 1392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:38.847717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56834 (local address 127.18.134.2:43259)
0619 02:53:38.847905 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:38.847910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:38.847926 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:53:38.848025 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:38.848029 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:38.848100 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:38.848214 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:53:38.848606 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.849307 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.850903 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.851342 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.851431 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:38.851946 (+ 515us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:53:38.852428 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:53:38.852526 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:38.852666 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 02:53:38.852730 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 02:53:38.860736 1332 raft_consensus.cc:1261] T 593c1ec8a0f7461da2f4fa79be5973dd P 083e444232ef4cc2a26fbee90b70b87d [term 1 FOLLOWER]: Refusing update from remote peer 011f109736ca46a0b59476e947fc20a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:53:38.860737 1069 raft_consensus.cc:1261] T 593c1ec8a0f7461da2f4fa79be5973dd P 8eb61eb911a24442acb8c3662c346f80 [term 1 FOLLOWER]: Refusing update from remote peer 011f109736ca46a0b59476e947fc20a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:53:38.861645 1402 consensus_queue.cc:1035] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "083e444232ef4cc2a26fbee90b70b87d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 44933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:53:38.861982 1400 consensus_queue.cc:1035] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eb61eb911a24442acb8c3662c346f80" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 34647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:53:38.871616 1407 mvcc.cc:205] Tried to move back new op lower bound from 7298406886844424192 to 7298406886727987200. Current Snapshot: MvccSnapshot[applied={T|T < 7298406886844424192}]
I20260619 02:53:38.872829 1408 mvcc.cc:205] Tried to move back new op lower bound from 7298406886844424192 to 7298406886727987200. Current Snapshot: MvccSnapshot[applied={T|T < 7298406886844424192}]
I20260619 02:53:38.886545 929 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:37190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:53:38.886826 929 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:37190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:53:38.890146 929 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91: Sending DeleteTablet for 3 replicas of tablet 593c1ec8a0f7461da2f4fa79be5973dd
I20260619 02:53:38.891278 1049 tablet_service.cc:1514] Processing DeleteTablet for tablet 593c1ec8a0f7461da2f4fa79be5973dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:53:38 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:36080
I20260619 02:53:38.891659 1181 tablet_service.cc:1514] Processing DeleteTablet for tablet 593c1ec8a0f7461da2f4fa79be5973dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:53:38 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:56818
I20260619 02:53:38.891957 1312 tablet_service.cc:1514] Processing DeleteTablet for tablet 593c1ec8a0f7461da2f4fa79be5973dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:53:38 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:57568
I20260619 02:53:38.892077 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 966
I20260619 02:53:38.895035 1419 tablet_replica.cc:330] T 593c1ec8a0f7461da2f4fa79be5973dd P 083e444232ef4cc2a26fbee90b70b87d: stopping tablet replica
I20260619 02:53:38.895429 1418 tablet_replica.cc:330] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3: stopping tablet replica
I20260619 02:53:38.895721 1419 raft_consensus.cc:2229] T 593c1ec8a0f7461da2f4fa79be5973dd P 083e444232ef4cc2a26fbee90b70b87d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:53:38.895888 1418 raft_consensus.cc:2229] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:53:38.896399 1419 raft_consensus.cc:2258] T 593c1ec8a0f7461da2f4fa79be5973dd P 083e444232ef4cc2a26fbee90b70b87d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:53:38.896904 1417 tablet_replica.cc:330] T 593c1ec8a0f7461da2f4fa79be5973dd P 8eb61eb911a24442acb8c3662c346f80: stopping tablet replica
I20260619 02:53:38.897254 1417 raft_consensus.cc:2229] T 593c1ec8a0f7461da2f4fa79be5973dd P 8eb61eb911a24442acb8c3662c346f80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:53:38.897636 1417 raft_consensus.cc:2258] T 593c1ec8a0f7461da2f4fa79be5973dd P 8eb61eb911a24442acb8c3662c346f80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:53:38.898447 1418 raft_consensus.cc:2258] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:53:38.899809 1417 ts_tablet_manager.cc:1905] T 593c1ec8a0f7461da2f4fa79be5973dd P 8eb61eb911a24442acb8c3662c346f80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:53:38.900346 1419 ts_tablet_manager.cc:1905] T 593c1ec8a0f7461da2f4fa79be5973dd P 083e444232ef4cc2a26fbee90b70b87d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:53:38.900887 1418 ts_tablet_manager.cc:1905] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:53:38.906792 1417 ts_tablet_manager.cc:1918] T 593c1ec8a0f7461da2f4fa79be5973dd P 8eb61eb911a24442acb8c3662c346f80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:53:38.906951 1417 log.cc:1196] T 593c1ec8a0f7461da2f4fa79be5973dd P 8eb61eb911a24442acb8c3662c346f80: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/wal/wals/593c1ec8a0f7461da2f4fa79be5973dd
I20260619 02:53:38.907248 1418 ts_tablet_manager.cc:1918] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:53:38.907413 1418 log.cc:1196] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/wal/wals/593c1ec8a0f7461da2f4fa79be5973dd
I20260619 02:53:38.907492 1417 ts_tablet_manager.cc:1939] T 593c1ec8a0f7461da2f4fa79be5973dd P 8eb61eb911a24442acb8c3662c346f80: Deleting consensus metadata
I20260619 02:53:38.907943 1418 ts_tablet_manager.cc:1939] T 593c1ec8a0f7461da2f4fa79be5973dd P 011f109736ca46a0b59476e947fc20a3: Deleting consensus metadata
I20260619 02:53:38.909474 916 catalog_manager.cc:4841] TS 8eb61eb911a24442acb8c3662c346f80 (127.18.134.1:34647): tablet 593c1ec8a0f7461da2f4fa79be5973dd (table test-table [id=c78d4053df434d1d86f5a3931d347746]) successfully deleted
I20260619 02:53:38.909544 1419 ts_tablet_manager.cc:1918] T 593c1ec8a0f7461da2f4fa79be5973dd P 083e444232ef4cc2a26fbee90b70b87d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:53:38.909582 913 catalog_manager.cc:4841] TS 011f109736ca46a0b59476e947fc20a3 (127.18.134.2:43259): tablet 593c1ec8a0f7461da2f4fa79be5973dd (table test-table [id=c78d4053df434d1d86f5a3931d347746]) successfully deleted
I20260619 02:53:38.909684 1419 log.cc:1196] T 593c1ec8a0f7461da2f4fa79be5973dd P 083e444232ef4cc2a26fbee90b70b87d: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/wal/wals/593c1ec8a0f7461da2f4fa79be5973dd
I20260619 02:53:38.910190 1419 ts_tablet_manager.cc:1939] T 593c1ec8a0f7461da2f4fa79be5973dd P 083e444232ef4cc2a26fbee90b70b87d: Deleting consensus metadata
I20260619 02:53:38.911051 1388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:38.892805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36096 (local address 127.18.134.1:34647)
0619 02:53:38.892985 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:38.892992 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:38.893099 (+ 107us) server_negotiation.cc:407] Connection header received
0619 02:53:38.893296 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:38.893301 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:38.893386 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:38.893486 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:38.895138 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.896184 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.898130 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:38.898741 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:38.898860 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:38.902177 (+ 3317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:38.902191 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:38.902207 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:38.902247 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:38.906163 (+ 3916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:38.907656 (+ 1493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:38.907665 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:38.907668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:38.907819 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:38.908344 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:38.908349 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:38.908351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:38.908842 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:38.909104 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:38.910425 (+ 1321us) server_negotiation.cc:299] Negotiation successful
0619 02:53:38.910693 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 02:53:38.911783 1109 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:53:38.911818 915 catalog_manager.cc:4841] TS 083e444232ef4cc2a26fbee90b70b87d (127.18.134.3:44933): tablet 593c1ec8a0f7461da2f4fa79be5973dd (table test-table [id=c78d4053df434d1d86f5a3931d347746]) successfully deleted
I20260619 02:53:38.999969 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 966
I20260619 02:53:39.014487 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 1118
I20260619 02:53:39.030788 1392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:39.015297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56842 (local address 127.18.134.2:43259)
0619 02:53:39.015494 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:39.015498 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:39.015520 (+ 22us) server_negotiation.cc:407] Connection header received
0619 02:53:39.015715 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:39.015719 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:39.015789 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:39.015868 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:39.017626 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:39.018506 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:39.020572 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:39.021076 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:39.021221 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:39.024549 (+ 3328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:39.024564 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:39.024581 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:39.024628 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:39.027683 (+ 3055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:39.028516 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:39.028525 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:39.028527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:39.028706 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:39.029116 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:39.029120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:39.029122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:39.029510 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:39.029835 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:39.030231 (+ 396us) server_negotiation.cc:299] Negotiation successful
0619 02:53:39.030403 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260619 02:53:39.031580 1241 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:53:39.144631 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 1118
I20260619 02:53:39.159886 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 1249
I20260619 02:53:39.173543 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:39.160653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57584 (local address 127.18.134.3:44933)
0619 02:53:39.160850 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:39.160856 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:39.160876 (+ 20us) server_negotiation.cc:407] Connection header received
0619 02:53:39.160937 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:39.160940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:39.161002 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:39.161108 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:39.162413 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:39.163173 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:39.164470 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:39.164881 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:39.164986 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:39.167820 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:39.167834 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:39.167850 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:39.167894 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:39.170924 (+ 3030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:39.171606 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:39.171614 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:39.171616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:39.171750 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:39.172101 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:39.172105 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:39.172108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:39.172458 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:39.172728 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:39.173008 (+ 280us) server_negotiation.cc:299] Negotiation successful
0619 02:53:39.173213 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 02:53:39.174145 1372 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:53:39.260660 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 1249
I20260619 02:53:39.273846 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 896
I20260619 02:53:39.275002 959 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:53:39.385841 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 896
I20260619 02:53:39.402676 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:33207
--webserver_interface=127.18.134.62
--webserver_port=34703
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:33207
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:53:39.713109 1424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:39.713400 1424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:39.713531 1424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:53:39.720849 1424 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:53:39.720958 1424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:39.721031 1424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:39.721105 1424 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:53:39.721171 1424 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:53:39.731221 1424 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:33207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:33207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=34703
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:39.732483 1424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:39.733718 1424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:53:39.744760 1430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:39.744843 1432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:39.745294 1429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:39.745592 1424 server_base.cc:1034] running on GCE node
I20260619 02:53:39.746341 1424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:39.747793 1424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:39.749009 1424 hybrid_clock.cc:648] HybridClock initialized: now 1781837619748949 us; error 51 us; skew 500 ppm
I20260619 02:53:39.749334 1424 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:39 dist-test-slave-9mr0 krb5kdc[345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837619, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:39.752396 1424 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:53:39.753785 1424 webserver.cc:460] Webserver started at http://127.18.134.62:34703/ using document root <none> and password file <none>
I20260619 02:53:39.754426 1424 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:39.754536 1424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:53:39.759716 1424 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260619 02:53:39.762396 1439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:39.763777 1424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 02:53:39.763936 1424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "6f92d2f13db7417da3983549b952dd91"
format_stamp: "Formatted at 2026-06-19 02:53:34 on dist-test-slave-9mr0"
server_key: "a666cd18c1b6770c6db96d1f4147971d"
server_key_iv: "8fa7a41f11bd84b00563b31988a41740"
server_key_version: "encryptionkey@0"
I20260619 02:53:39.764926 1424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:39.781219 1424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:39.785432 1424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:39.785682 1424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:39.803002 1424 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:33207
I20260619 02:53:39.803025 1490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:33207 every 8 connection(s)
I20260619 02:53:39.804469 1424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:53:39.811236 1491 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:53:39.812397 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 1424
I20260619 02:53:39.813360 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:34647
--local_ip_for_outbound_sockets=127.18.134.1
--tserver_master_addrs=127.18.134.62:33207
--webserver_port=36467
--webserver_interface=127.18.134.1
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:53:39.815629 1491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91: Bootstrap starting.
I20260619 02:53:39.836717 1491 log.cc:824] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91: Log is configured to *not* fsync() on all Append() calls
I20260619 02:53:39.853978 1491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:53:39.854481 1491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91: Bootstrap complete.
I20260619 02:53:39.863360 1491 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f92d2f13db7417da3983549b952dd91" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33207 } }
I20260619 02:53:39.864234 1491 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f92d2f13db7417da3983549b952dd91, State: Initialized, Role: FOLLOWER
I20260619 02:53:39.864857 1491 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f92d2f13db7417da3983549b952dd91" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33207 } }
I20260619 02:53:39.865067 1491 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:53:39.865176 1491 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:53:39.865334 1491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [term 2 FOLLOWER]: Advancing to term 3
I20260619 02:53:39.869184 1491 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f92d2f13db7417da3983549b952dd91" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33207 } }
I20260619 02:53:39.869688 1491 leader_election.cc:302] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f92d2f13db7417da3983549b952dd91; no voters:
I20260619 02:53:39.870328 1491 leader_election.cc:288] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 02:53:39.870461 1495 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 02:53:39.871546 1495 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [term 3 LEADER]: Becoming Leader. State: Replica: 6f92d2f13db7417da3983549b952dd91, State: Running, Role: LEADER
I20260619 02:53:39.872051 1495 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f92d2f13db7417da3983549b952dd91" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33207 } }
I20260619 02:53:39.872371 1491 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:53:39.874406 1496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6f92d2f13db7417da3983549b952dd91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f92d2f13db7417da3983549b952dd91" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33207 } } }
I20260619 02:53:39.874763 1496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [sys.catalog]: This master's current role is: LEADER
I20260619 02:53:39.875361 1497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f92d2f13db7417da3983549b952dd91. Latest consensus state: current_term: 3 leader_uuid: "6f92d2f13db7417da3983549b952dd91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f92d2f13db7417da3983549b952dd91" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 33207 } } }
I20260619 02:53:39.875707 1497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f92d2f13db7417da3983549b952dd91 [sys.catalog]: This master's current role is: LEADER
I20260619 02:53:39.879359 1500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:53:39.887872 1500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:53:39.895581 1500 catalog_manager.cc:1259] Loaded cluster ID: c8aa49fb650f439e845d95a09df7e6f9
I20260619 02:53:39.895709 1500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260619 02:53:39.898103 1500 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ff8ac8bf9ad google::LogMessage::Fail() at ??:0
@ 0x7ff8ac8c288a google::LogMessage::SendToLog() at ??:0
@ 0x7ff8ac8bf4de google::LogMessage::Flush() at ??:0
@ 0x7ff8ac8c0a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ff8c22f2512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7ff8c22ef976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7ff8c2377610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7ff8c2377439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7ff8af50b345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7ff8af50e91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7ff8af50e799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7ff8af4e9eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7ff8b27c36db start_thread at ??:0
@ 0x7ff8ab75961f clone at ??:0
W20260619 02:53:40.161059 1493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:40.161391 1493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:40.161553 1493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:53:40.169144 1493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:40.169283 1493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:40.169458 1493 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:53:40.180250 1493 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:34647
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=36467
--tserver_master_addrs=127.18.134.62:33207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:40.181581 1493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:40.183060 1493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:53:40.196309 1533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:40.196345 1535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:40.196367 1532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:40.197423 1493 server_base.cc:1034] running on GCE node
I20260619 02:53:40.198035 1493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:40.199029 1493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:40.200248 1493 hybrid_clock.cc:648] HybridClock initialized: now 1781837620200201 us; error 40 us; skew 500 ppm
I20260619 02:53:40.200546 1493 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:40 dist-test-slave-9mr0 krb5kdc[345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837620, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:40.203647 1493 init.cc:377] Logged in from keytab as kudu/127.18.134.1@KRBTEST.COM (short username kudu)
I20260619 02:53:40.205047 1493 webserver.cc:460] Webserver started at http://127.18.134.1:36467/ using document root <none> and password file <none>
I20260619 02:53:40.205719 1493 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:40.205825 1493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:53:40.211148 1493 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260619 02:53:40.213884 1542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:40.215373 1493 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:53:40.215526 1493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "8eb61eb911a24442acb8c3662c346f80"
format_stamp: "Formatted at 2026-06-19 02:53:34 on dist-test-slave-9mr0"
server_key: "650df52c8f80a9b31296004d51d0b22a"
server_key_iv: "2632c9cb4bc69fab6feab44663ff1d34"
server_key_version: "encryptionkey@0"
I20260619 02:53:40.216503 1493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:40.239338 1493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:40.243628 1493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:40.243888 1493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:40.244885 1493 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:53:40.246275 1493 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:53:40.246376 1493 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:40.246482 1493 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:53:40.246559 1493 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:40.276839 1493 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:34647
I20260619 02:53:40.276933 1654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:34647 every 8 connection(s)
I20260619 02:53:40.278296 1493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:53:40.282974 1657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:40.281096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.1:54681)
0619 02:53:40.281836 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:40.282069 (+ 233us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":272,"threads_started":1}
W20260619 02:53:40.283672 1655 heartbeater.cc:644] Failed to heartbeat to 127.18.134.62:33207 (0 consecutive failures): Network error: Failed to ping master at 127.18.134.62:33207: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
I20260619 02:53:40.284452 1657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:40.284139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.1:54669)
0619 02:53:40.284277 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:40.284341 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 02:53:40.285368 1657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:40.284970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.1:53827)
0619 02:53:40.285121 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:40.285199 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260619 02:53:40.285597 1655 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 02:53:40.287490 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 1493
I20260619 02:53:40.288050 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:43259
--local_ip_for_outbound_sockets=127.18.134.2
--tserver_master_addrs=127.18.134.62:33207
--webserver_port=42729
--webserver_interface=127.18.134.2
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:53:40.604727 1658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:40.605027 1658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:40.605171 1658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:53:40.612485 1658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:40.612623 1658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:40.612783 1658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:53:40.622747 1658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:43259
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=42729
--tserver_master_addrs=127.18.134.62:33207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:40.624042 1658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:40.625362 1658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:53:40.638247 1666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:40.638356 1664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:40.638176 1663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:40.639448 1658 server_base.cc:1034] running on GCE node
I20260619 02:53:40.640100 1658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:40.641052 1658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:40.642257 1658 hybrid_clock.cc:648] HybridClock initialized: now 1781837620642209 us; error 43 us; skew 500 ppm
I20260619 02:53:40.642552 1658 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:40 dist-test-slave-9mr0 krb5kdc[345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837620, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:40.645462 1658 init.cc:377] Logged in from keytab as kudu/127.18.134.2@KRBTEST.COM (short username kudu)
I20260619 02:53:40.646858 1658 webserver.cc:460] Webserver started at http://127.18.134.2:42729/ using document root <none> and password file <none>
I20260619 02:53:40.647496 1658 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:40.647615 1658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:53:40.652863 1658 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260619 02:53:40.657346 1673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:40.658754 1658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:53:40.658911 1658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "011f109736ca46a0b59476e947fc20a3"
format_stamp: "Formatted at 2026-06-19 02:53:35 on dist-test-slave-9mr0"
server_key: "794a5cd224ed315bd923c3e7fdcde710"
server_key_iv: "76c8d3f264a2dc32f478810894caf48e"
server_key_version: "encryptionkey@0"
I20260619 02:53:40.659874 1658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:40.675864 1658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:40.680027 1658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:40.680294 1658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:40.681326 1658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:53:40.682682 1658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:53:40.682777 1658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:40.682884 1658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:53:40.682958 1658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:40.713235 1658 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:43259
I20260619 02:53:40.713325 1785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:43259 every 8 connection(s)
I20260619 02:53:40.714733 1658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:53:40.718595 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 1658
I20260619 02:53:40.719343 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:44933
--local_ip_for_outbound_sockets=127.18.134.3
--tserver_master_addrs=127.18.134.62:33207
--webserver_port=43619
--webserver_interface=127.18.134.3
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:53:40.719906 1788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:40.717652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.2:59243)
0619 02:53:40.718443 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:40.718736 (+ 293us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":271,"threads_started":1}
W20260619 02:53:40.720680 1786 heartbeater.cc:644] Failed to heartbeat to 127.18.134.62:33207 (0 consecutive failures): Network error: Failed to ping master at 127.18.134.62:33207: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
I20260619 02:53:40.721572 1788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:40.721211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.2:33071)
0619 02:53:40.721370 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:40.721449 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260619 02:53:40.722414 1788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:40.722079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.2:56549)
0619 02:53:40.722208 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:40.722293 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260619 02:53:40.722656 1786 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:53:41.040534 1789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:41.040817 1789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:41.040935 1789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:53:41.048255 1789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:41.048372 1789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:41.048519 1789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:53:41.060290 1789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:44933
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=43619
--tserver_master_addrs=127.18.134.62:33207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:41.061913 1789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:41.063623 1789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:53:41.076598 1795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:41.076663 1794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:41.077240 1797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:41.077394 1789 server_base.cc:1034] running on GCE node
I20260619 02:53:41.077986 1789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:41.078912 1789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:41.080101 1789 hybrid_clock.cc:648] HybridClock initialized: now 1781837621080051 us; error 44 us; skew 500 ppm
I20260619 02:53:41.080410 1789 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:41 dist-test-slave-9mr0 krb5kdc[345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837621, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:41.086012 1789 init.cc:377] Logged in from keytab as kudu/127.18.134.3@KRBTEST.COM (short username kudu)
I20260619 02:53:41.087440 1789 webserver.cc:460] Webserver started at http://127.18.134.3:43619/ using document root <none> and password file <none>
I20260619 02:53:41.088075 1789 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:41.088173 1789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:53:41.093354 1789 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260619 02:53:41.096091 1804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:41.097369 1789 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260619 02:53:41.097505 1789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "083e444232ef4cc2a26fbee90b70b87d"
format_stamp: "Formatted at 2026-06-19 02:53:35 on dist-test-slave-9mr0"
server_key: "582892f9e0421d7d602e1303a4c62d90"
server_key_iv: "194c56670a55ce2eaf708d1617bac467"
server_key_version: "encryptionkey@0"
I20260619 02:53:41.098443 1789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:41.119292 1789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:41.123291 1789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:41.123530 1789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:41.124444 1789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:53:41.125722 1789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:53:41.125813 1789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:41.125916 1789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:53:41.125981 1789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:41.155869 1789 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:44933
I20260619 02:53:41.155949 1916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:44933 every 8 connection(s)
I20260619 02:53:41.157305 1789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:53:41.161837 1919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:41.160052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.3:59095)
0619 02:53:41.160788 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:41.161003 (+ 215us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":283,"threads_started":1}
W20260619 02:53:41.162505 1917 heartbeater.cc:644] Failed to heartbeat to 127.18.134.62:33207 (0 consecutive failures): Network error: Failed to ping master at 127.18.134.62:33207: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
I20260619 02:53:41.163398 1919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:41.163057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.3:58505)
0619 02:53:41.163220 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:41.163293 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260619 02:53:41.163573 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 1789
I20260619 02:53:41.164268 1919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:41.163880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.3:57661)
0619 02:53:41.164063 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:41.164154 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260619 02:53:41.164512 1917 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 02:53:41.286973 1921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:41.286129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.1:39669)
0619 02:53:41.286624 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:41.286752 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":226,"threads_started":1}
I20260619 02:53:41.723920 1922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:41.723162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.2:46235)
0619 02:53:41.723592 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:41.723703 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":205,"threads_started":1}
I20260619 02:53:42.165944 1923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:42.165059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.3:54287)
0619 02:53:42.165569 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:42.165685 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":237,"threads_started":1}
I20260619 02:53:42.288661 1924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:42.287869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.1:45765)
0619 02:53:42.288396 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:42.288494 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":219,"threads_started":1}
I20260619 02:53:42.725495 1925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:42.724806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.2:51789)
0619 02:53:42.725246 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:42.725333 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":212,"threads_started":1}
I20260619 02:53:43.167663 1926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:43.166948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.3:39735)
0619 02:53:43.167409 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:43.167491 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":226,"threads_started":1}
I20260619 02:53:43.290329 1927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:43.289561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.1:56515)
0619 02:53:43.290022 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:43.290151 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":236,"threads_started":1}
I20260619 02:53:43.727304 1928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:43.726442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.2:38563)
0619 02:53:43.727018 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:43.727124 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":313,"threads_started":1}
I20260619 02:53:44.169157 1929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:44.168474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.3:53357)
0619 02:53:44.168901 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:44.168983 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":220,"threads_started":1}
I20260619 02:53:44.291975 1930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:44.291259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.1:51385)
0619 02:53:44.291687 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:44.291784 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":193,"threads_started":1}
I20260619 02:53:44.729161 1931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:44.728293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.2:33771)
0619 02:53:44.728788 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:44.728899 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":241,"threads_started":1}
I20260619 02:53:45.170886 1932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:45.170096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.3:40599)
0619 02:53:45.170548 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:45.170671 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":221,"threads_started":1}
I20260619 02:53:45.293562 1933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:45.292805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.1:52307)
0619 02:53:45.293310 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:45.293394 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":231,"threads_started":1}
I20260619 02:53:45.730721 1934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:45.729997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.2:53401)
0619 02:53:45.730434 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:45.730513 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":223,"threads_started":1}
I20260619 02:53:46.172708 1935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:46.171835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.3:41059)
0619 02:53:46.172368 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:46.172474 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":249,"threads_started":1}
I20260619 02:53:46.295222 1936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:46.294486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.1:36237)
0619 02:53:46.294972 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:46.295063 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":263,"threads_started":1}
I20260619 02:53:46.732468 1937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:46.731612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.2:57459)
0619 02:53:46.732182 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:46.732285 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":258,"threads_started":1}
I20260619 02:53:47.174240 1938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:47.173524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.3:44457)
0619 02:53:47.173977 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:47.174069 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":229,"threads_started":1}
I20260619 02:53:47.296990 1939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:47.296204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.1:43019)
0619 02:53:47.296697 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:47.296782 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":210,"threads_started":1}
I20260619 02:53:47.734110 1940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:47.733350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.2:49513)
0619 02:53:47.733822 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:47.733904 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":209,"threads_started":1}
I20260619 02:53:48.176101 1941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:48.175279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.3:37225)
0619 02:53:48.175798 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:48.175885 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":224,"threads_started":1}
I20260619 02:53:48.298776 1942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:48.297985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.1:47191)
0619 02:53:48.298489 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:48.298577 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":282,"threads_started":1}
I20260619 02:53:48.735710 1943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:48.735027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.2:59635)
0619 02:53:48.735469 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:48.735552 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":215,"threads_started":1}
I20260619 02:53:49.177867 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:49.177008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.3:35915)
0619 02:53:49.177535 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:49.177641 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":292,"threads_started":1}
I20260619 02:53:49.300343 1945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:49.299651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.1:54023)
0619 02:53:49.300094 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:49.300174 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":211,"threads_started":1}
I20260619 02:53:49.737253 1946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:49.736513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.2:39045)
0619 02:53:49.736967 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:49.737045 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":221,"threads_started":1}
I20260619 02:53:50.179502 1947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:50.178744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.3:37589)
0619 02:53:50.179241 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:50.179327 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":213,"threads_started":1}
I20260619 02:53:50.301968 1948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:50.301291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.1:34235)
0619 02:53:50.301722 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:50.301809 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":223,"threads_started":1}
I20260619 02:53:50.738988 1949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:50.738190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.2:54409)
0619 02:53:50.738732 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:50.738822 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":278,"threads_started":1}
I20260619 02:53:51.181150 1950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:51.180420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.3:44187)
0619 02:53:51.180852 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:51.180935 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":231,"threads_started":1}
I20260619 02:53:51.303618 1951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:51.302881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.1:40987)
0619 02:53:51.303347 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:51.303443 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":249,"threads_started":1}
I20260619 02:53:51.740610 1952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:51.739915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.2:34953)
0619 02:53:51.740326 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:51.740418 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":196,"threads_started":1}
I20260619 02:53:52.182894 1953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:52.182125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.3:44229)
0619 02:53:52.182564 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:52.182690 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":201,"threads_started":1}
I20260619 02:53:52.305202 1954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:52.304469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.1:59643)
0619 02:53:52.304916 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:52.305042 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":221,"threads_started":1}
I20260619 02:53:52.742158 1955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:52.741467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.2:58943)
0619 02:53:52.741929 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:52.742006 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":231,"threads_started":1}
I20260619 02:53:53.184705 1956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:53.183914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.3:46119)
0619 02:53:53.184440 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:53.184539 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":282,"threads_started":1}
I20260619 02:53:53.306993 1957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:53.306140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.1:35471)
0619 02:53:53.306678 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:53.306798 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":231,"threads_started":1}
I20260619 02:53:53.743824 1958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:53.743102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.2:39519)
0619 02:53:53.743547 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:53.743643 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":210,"threads_started":1}
I20260619 02:53:54.186431 1959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:54.185648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.3:45133)
0619 02:53:54.186147 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:54.186242 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":235,"threads_started":1}
I20260619 02:53:54.308599 1960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:54.307797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.1:35813)
0619 02:53:54.308302 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:54.308406 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":263,"threads_started":1}
I20260619 02:53:54.745519 1961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:54.744777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.2:50663)
0619 02:53:54.745247 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:54.745343 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":244,"threads_started":1}
I20260619 02:53:55.188060 1962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:55.187332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.3:45523)
0619 02:53:55.187794 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:55.187872 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":253,"threads_started":1}
I20260619 02:53:55.310556 1963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:55.309753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.1:35475)
0619 02:53:55.310264 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:55.310377 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":306,"threads_started":1}
I20260619 02:53:55.747567 1964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:55.746784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.2:59365)
0619 02:53:55.747260 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:55.747360 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":250,"threads_started":1}
I20260619 02:53:56.166806 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 1493
I20260619 02:53:56.186304 1965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:56.167937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41018 (local address 127.18.134.1:34647)
0619 02:53:56.169462 (+ 1525us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:56.169474 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:56.169513 (+ 39us) server_negotiation.cc:407] Connection header received
0619 02:53:56.171590 (+ 2077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:56.171600 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:56.172057 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:56.172352 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:56.174161 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:56.175453 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:56.177219 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:56.177891 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:56.178050 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:56.180558 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:56.180566 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:56.180581 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:56.180625 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:56.183443 (+ 2818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:56.184070 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:56.184077 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:56.184083 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:56.184261 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:56.184540 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:56.184543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:56.184545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:56.185062 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:56.185288 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:56.185616 (+ 328us) server_negotiation.cc:299] Negotiation successful
0619 02:53:56.185997 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1358,"thread_start_us":1241,"threads_started":1}
I20260619 02:53:56.187610 1650 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:53:56.190022 1967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:56.189186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:33207 (local address 127.18.134.3:35597)
0619 02:53:56.189663 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:56.189816 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:33207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":224,"threads_started":1}
I20260619 02:53:56.271916 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 1493
I20260619 02:53:56.286031 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 1658
I20260619 02:53:56.303215 1968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:56.286945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60344 (local address 127.18.134.2:43259)
0619 02:53:56.287501 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:56.287516 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:56.287553 (+ 37us) server_negotiation.cc:407] Connection header received
0619 02:53:56.289135 (+ 1582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:56.289145 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:56.289558 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:56.289890 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:56.291376 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:56.292590 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:56.293813 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:56.294473 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:56.294683 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:56.297175 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:56.297183 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:56.297200 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:56.297240 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:56.299878 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:56.300600 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:56.300607 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:56.300621 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:56.300787 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:56.301139 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:56.301143 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:56.301145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:56.301683 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:56.301991 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:56.302448 (+ 457us) server_negotiation.cc:299] Negotiation successful
0619 02:53:56.302933 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":266,"threads_started":1}
I20260619 02:53:56.304579 1781 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:53:56.387441 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 1658
I20260619 02:53:56.401633 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 1789
I20260619 02:53:56.419612 1970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:56.402592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33144 (local address 127.18.134.3:44933)
0619 02:53:56.403243 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:56.403256 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:56.403296 (+ 40us) server_negotiation.cc:407] Connection header received
0619 02:53:56.404894 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:56.404902 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:56.405300 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:56.405610 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:56.407102 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:56.408324 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:56.409534 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:56.410183 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:56.410353 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:56.413335 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:56.413344 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:56.413368 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:56.413425 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:56.416460 (+ 3035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:56.417138 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:56.417145 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:56.417151 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:56.417312 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:56.417679 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:56.417683 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:56.417685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:56.418199 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:56.418502 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:56.418928 (+ 426us) server_negotiation.cc:299] Negotiation successful
0619 02:53:56.419324 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":260,"threads_started":1}
I20260619 02:53:56.420964 1912 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:53:56.501458 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 1789
2026-06-19T02:53:56Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (23694 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:53:56 dist-test-slave-9mr0 krb5kdc[1974](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:53:56 dist-test-slave-9mr0 krb5kdc[1974](info): set up 2 sockets
Jun 19 02:53:56 dist-test-slave-9mr0 krb5kdc[1974](info): commencing operation
krb5kdc: starting...
W20260619 02:53:57.507207 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.970s user 0.006s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:53:57 dist-test-slave-9mr0 krb5kdc[1974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:53:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:53:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:53:57.718187 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:35511
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:45187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:35511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:53:58.034518 1987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:58.034835 1987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:58.035008 1987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:53:58.042196 1987 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 02:53:58.042289 1987 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 02:53:58.042328 1987 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 02:53:58.042402 1987 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:53:58.042469 1987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:58.042564 1987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:58.042618 1987 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:53:58.042711 1987 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:53:58.052884 1987 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:45187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:35511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:35511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:58.054141 1987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:58.055460 1987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:53:58.066841 1992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:58.066877 1995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:58.066982 1993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:58.067576 1987 server_base.cc:1034] running on GCE node
I20260619 02:53:58.068325 1987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:58.069703 1987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:58.070914 1987 hybrid_clock.cc:648] HybridClock initialized: now 1781837638070843 us; error 42 us; skew 500 ppm
I20260619 02:53:58.071259 1987 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:58 dist-test-slave-9mr0 krb5kdc[1974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837638, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:58.074345 1987 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:53:58.075887 1987 webserver.cc:460] Webserver started at http://127.18.134.62:45413/ using document root <none> and password file <none>
I20260619 02:53:58.076563 1987 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:58.076661 1987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:53:58.076952 1987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:53:58.079882 1987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "2d04b4e2748641509554c98f7efecfc3"
format_stamp: "Formatted at 2026-06-19 02:53:58 on dist-test-slave-9mr0"
server_key: "c5584d847587e46c08071fd2eb906d26"
server_key_iv: "43001b69266fabf6ed8d4ca2444ee469"
server_key_version: "encryptionkey@0"
I20260619 02:53:58.080626 1987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "2d04b4e2748641509554c98f7efecfc3"
format_stamp: "Formatted at 2026-06-19 02:53:58 on dist-test-slave-9mr0"
server_key: "c5584d847587e46c08071fd2eb906d26"
server_key_iv: "43001b69266fabf6ed8d4ca2444ee469"
server_key_version: "encryptionkey@0"
I20260619 02:53:58.085719 1987 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260619 02:53:58.089102 2002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:58.090377 1987 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:53:58.090541 1987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "2d04b4e2748641509554c98f7efecfc3"
format_stamp: "Formatted at 2026-06-19 02:53:58 on dist-test-slave-9mr0"
server_key: "c5584d847587e46c08071fd2eb906d26"
server_key_iv: "43001b69266fabf6ed8d4ca2444ee469"
server_key_version: "encryptionkey@0"
I20260619 02:53:58.090811 1987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:58.112469 1987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:58.116597 1987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:58.116843 1987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:58.134074 1987 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:35511
I20260619 02:53:58.134092 2053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:35511 every 8 connection(s)
I20260619 02:53:58.135340 1987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:53:58.139108 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 1987
I20260619 02:53:58.139196 2054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:53:58.139302 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:53:58.139921 18968 external_mini_cluster.cc:1428] Setting key ef7267ae5fadce46222d35f8c1ba470c
I20260619 02:53:58.153721 2054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3: Bootstrap starting.
Jun 19 02:53:58 dist-test-slave-9mr0 krb5kdc[1974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:53:58.157789 2054 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3: Neither blocks nor log segments found. Creating new log.
I20260619 02:53:58.159268 2054 log.cc:824] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3: Log is configured to *not* fsync() on all Append() calls
I20260619 02:53:58.162914 2054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3: No bootstrap required, opened a new log
I20260619 02:53:58.164601 2057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:58.141783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32958 (local address 127.18.134.62:35511)
0619 02:53:58.142647 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:58.142669 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:58.142728 (+ 59us) server_negotiation.cc:407] Connection header received
0619 02:53:58.144444 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:58.144473 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:58.145025 (+ 552us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:58.145481 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:58.146744 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:58.148720 (+ 1976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:58.150103 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:58.151241 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:58.151468 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:58.155591 (+ 4123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:58.155604 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:58.155623 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:58.155673 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:58.159776 (+ 4103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:58.160728 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:58.160735 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:58.160740 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:58.161019 (+ 279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:58.161406 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:58.161411 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:58.161414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:58.162143 (+ 729us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:58.162450 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:58.162860 (+ 410us) server_negotiation.cc:299] Negotiation successful
0619 02:53:58.163308 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":324,"threads_started":1}
I20260619 02:53:58.171482 2054 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d04b4e2748641509554c98f7efecfc3" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35511 } }
I20260619 02:53:58.171800 2054 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:53:58.171888 2054 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d04b4e2748641509554c98f7efecfc3, State: Initialized, Role: FOLLOWER
I20260619 02:53:58.172386 2054 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d04b4e2748641509554c98f7efecfc3" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35511 } }
I20260619 02:53:58.172606 2054 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:53:58.172700 2054 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:53:58.172818 2054 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:53:58.174912 2054 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d04b4e2748641509554c98f7efecfc3" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35511 } }
I20260619 02:53:58.175357 2054 leader_election.cc:302] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d04b4e2748641509554c98f7efecfc3; no voters:
I20260619 02:53:58.175873 2054 leader_election.cc:288] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:53:58.176014 2059 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:53:58.177091 2059 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [term 1 LEADER]: Becoming Leader. State: Replica: 2d04b4e2748641509554c98f7efecfc3, State: Running, Role: LEADER
I20260619 02:53:58.177600 2059 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d04b4e2748641509554c98f7efecfc3" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35511 } }
I20260619 02:53:58.177974 2054 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:53:58.180365 2061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d04b4e2748641509554c98f7efecfc3. Latest consensus state: current_term: 1 leader_uuid: "2d04b4e2748641509554c98f7efecfc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d04b4e2748641509554c98f7efecfc3" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35511 } } }
I20260619 02:53:58.180509 2060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d04b4e2748641509554c98f7efecfc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d04b4e2748641509554c98f7efecfc3" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35511 } } }
I20260619 02:53:58.181070 2060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [sys.catalog]: This master's current role is: LEADER
I20260619 02:53:58.181072 2061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [sys.catalog]: This master's current role is: LEADER
I20260619 02:53:58.183812 2067 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:53:58.188901 2067 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:53:58.197676 2067 catalog_manager.cc:1347] Generated new cluster ID: 070be589d2284f839da167d7b368c425
I20260619 02:53:58.197790 2067 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:53:58.220625 2067 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:53:58.222558 2067 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:53:58.238929 2067 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3: Generated new TSK 0
I20260619 02:53:58.240070 2067 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.134.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
I20260619 02:53:58.290194 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:0
--local_ip_for_outbound_sockets=127.18.134.1
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:35511
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:45187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:53:58.604687 2080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:58.604952 2080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:58.605053 2080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:53:58.612138 2080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:58.612246 2080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:58.612356 2080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:53:58.621906 2080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:45187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:35511
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:58.623133 2080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:58.624369 2080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:53:58.636816 2085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:58.636766 2086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:58.636981 2088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:58.637926 2080 server_base.cc:1034] running on GCE node
I20260619 02:53:58.638505 2080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:58.639424 2080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:58.640623 2080 hybrid_clock.cc:648] HybridClock initialized: now 1781837638640561 us; error 53 us; skew 500 ppm
I20260619 02:53:58.640924 2080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:58 dist-test-slave-9mr0 krb5kdc[1974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837638, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:58.643894 2080 init.cc:377] Logged in from keytab as kudu/127.18.134.1@KRBTEST.COM (short username kudu)
I20260619 02:53:58.645242 2080 webserver.cc:460] Webserver started at http://127.18.134.1:38429/ using document root <none> and password file <none>
I20260619 02:53:58.645867 2080 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:58.645961 2080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:53:58.646211 2080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:53:58.648952 2080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/data/instance:
uuid: "2da3d0dd4eb1432787df0d9838ba8b07"
format_stamp: "Formatted at 2026-06-19 02:53:58 on dist-test-slave-9mr0"
server_key: "3ba52d3a1c4e3e634d225f1a0b82e3ff"
server_key_iv: "0d4817d0a8d4f845f93f57ad7aaa1564"
server_key_version: "encryptionkey@0"
I20260619 02:53:58.649627 2080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance:
uuid: "2da3d0dd4eb1432787df0d9838ba8b07"
format_stamp: "Formatted at 2026-06-19 02:53:58 on dist-test-slave-9mr0"
server_key: "3ba52d3a1c4e3e634d225f1a0b82e3ff"
server_key_iv: "0d4817d0a8d4f845f93f57ad7aaa1564"
server_key_version: "encryptionkey@0"
I20260619 02:53:58.654520 2080 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.007s
I20260619 02:53:58.657894 2095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:58.659322 2080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:53:58.659471 2080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "2da3d0dd4eb1432787df0d9838ba8b07"
format_stamp: "Formatted at 2026-06-19 02:53:58 on dist-test-slave-9mr0"
server_key: "3ba52d3a1c4e3e634d225f1a0b82e3ff"
server_key_iv: "0d4817d0a8d4f845f93f57ad7aaa1564"
server_key_version: "encryptionkey@0"
I20260619 02:53:58.659641 2080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:58.682601 2080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:58.686913 2080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:58.687162 2080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:58.688023 2080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:53:58.689320 2080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:53:58.689391 2080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:58.689471 2080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:53:58.689522 2080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:58.719611 2080 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:41291
I20260619 02:53:58.719687 2207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:41291 every 8 connection(s)
I20260619 02:53:58.720907 2080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:53:58.721400 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 2080
I20260619 02:53:58.721593 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance
I20260619 02:53:58.722390 18968 external_mini_cluster.cc:1428] Setting key 118f0710366414496708753021a8c9d5
Jun 19 02:53:58 dist-test-slave-9mr0 krb5kdc[1974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837638, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:53:58.740815 2210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:58.723586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:56649 (local address 127.18.134.62:35511)
0619 02:53:58.724016 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:58.724021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:58.725190 (+ 1169us) server_negotiation.cc:407] Connection header received
0619 02:53:58.727234 (+ 2044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:58.727240 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:58.727329 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:58.727431 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:58.729193 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:58.729896 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:58.731497 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:58.731898 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:58.732010 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:58.735342 (+ 3332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:58.735347 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:58.735350 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:58.735384 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:58.737764 (+ 2380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:58.738505 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:58.738511 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:58.738514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:58.738704 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:58.739133 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:58.739138 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:58.739141 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:58.739418 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:58.739562 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:58.740226 (+ 664us) server_negotiation.cc:299] Negotiation successful
0619 02:53:58.740446 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":211,"threads_started":1}
I20260619 02:53:58.741793 2211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:58.724181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.1:56649)
0619 02:53:58.725027 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:58.725074 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:58.726941 (+ 1867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:58.727700 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:58.727709 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:58.728419 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:53:58.729002 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:58.729020 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:58.730055 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:58.730060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:58.731354 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:58.731366 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:58.732133 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:58.732140 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:58.732349 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:58.732983 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:53:58.733005 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:53:58.735127 (+ 2122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:53:58.737899 (+ 2772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:58.737908 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:58.737933 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:58.738347 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:58.738817 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:58.738821 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:58.738823 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:58.739009 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:58.739548 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:53:58.739556 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:53:58.739882 (+ 326us) client_negotiation.cc:769] Sending connection context
0619 02:53:58.740113 (+ 231us) client_negotiation.cc:240] Negotiation successful
0619 02:53:58.740648 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":330,"threads_started":1}
WARNING: no policy specified for kudu/127.18.134.2@KRBTEST.COM; defaulting to no policy
I20260619 02:53:58.743292 2208 heartbeater.cc:347] Connected to a master server at 127.18.134.62:35511
I20260619 02:53:58.743571 2208 heartbeater.cc:464] Registering TS with master...
I20260619 02:53:58.744235 2208 heartbeater.cc:511] Master 127.18.134.62:35511 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.2@KRBTEST.COM" created.
I20260619 02:53:58.746488 2019 ts_manager.cc:194] Registered new tserver with Master: 2da3d0dd4eb1432787df0d9838ba8b07 (127.18.134.1:41291)
I20260619 02:53:58.748277 2019 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.1@KRBTEST.COM'} at 127.18.134.1:56649
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
I20260619 02:53:58.773350 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:0
--local_ip_for_outbound_sockets=127.18.134.2
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:35511
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:45187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:53:59.095242 2214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:59.095578 2214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:59.095695 2214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:53:59.103173 2214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:59.103297 2214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:59.103454 2214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:53:59.113466 2214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:45187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:35511
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:59.114859 2214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:59.116180 2214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:53:59.129618 2222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:59.129562 2220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:59.129652 2219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:59.130291 2214 server_base.cc:1034] running on GCE node
I20260619 02:53:59.130923 2214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:59.131862 2214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:59.133091 2214 hybrid_clock.cc:648] HybridClock initialized: now 1781837639133034 us; error 49 us; skew 500 ppm
I20260619 02:53:59.133455 2214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:59 dist-test-slave-9mr0 krb5kdc[1974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837639, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:59.136595 2214 init.cc:377] Logged in from keytab as kudu/127.18.134.2@KRBTEST.COM (short username kudu)
I20260619 02:53:59.138168 2214 webserver.cc:460] Webserver started at http://127.18.134.2:44289/ using document root <none> and password file <none>
I20260619 02:53:59.138898 2214 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:59.139011 2214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:53:59.139290 2214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:53:59.142210 2214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/data/instance:
uuid: "8451d39ceeb948e0814cbdc529dae4aa"
format_stamp: "Formatted at 2026-06-19 02:53:59 on dist-test-slave-9mr0"
server_key: "274da2bed2cf9d78eb59163bf4114c6b"
server_key_iv: "9b89f349b3427a91c1f59b0d3854a4ef"
server_key_version: "encryptionkey@0"
I20260619 02:53:59.142962 2214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance:
uuid: "8451d39ceeb948e0814cbdc529dae4aa"
format_stamp: "Formatted at 2026-06-19 02:53:59 on dist-test-slave-9mr0"
server_key: "274da2bed2cf9d78eb59163bf4114c6b"
server_key_iv: "9b89f349b3427a91c1f59b0d3854a4ef"
server_key_version: "encryptionkey@0"
I20260619 02:53:59.148248 2214 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 02:53:59.151737 2229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:59.153076 2214 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:53:59.153223 2214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "8451d39ceeb948e0814cbdc529dae4aa"
format_stamp: "Formatted at 2026-06-19 02:53:59 on dist-test-slave-9mr0"
server_key: "274da2bed2cf9d78eb59163bf4114c6b"
server_key_iv: "9b89f349b3427a91c1f59b0d3854a4ef"
server_key_version: "encryptionkey@0"
I20260619 02:53:59.153409 2214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:59.180523 2214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:59.184888 2214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:59.185153 2214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:59.186087 2214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:53:59.187433 2214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:53:59.187513 2214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:59.187630 2214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:53:59.187678 2214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:59.216300 2214 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:37295
I20260619 02:53:59.216357 2341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:37295 every 8 connection(s)
I20260619 02:53:59.217504 2214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:53:59.226068 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 2214
I20260619 02:53:59.226241 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance
I20260619 02:53:59.226696 18968 external_mini_cluster.cc:1428] Setting key 0d678894f8e5b752c1733c11de3b6641
Jun 19 02:53:59 dist-test-slave-9mr0 krb5kdc[1974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837639, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:53:59.237090 2210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:59.219951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:33337 (local address 127.18.134.62:35511)
0619 02:53:59.220124 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:59.220129 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:59.221405 (+ 1276us) server_negotiation.cc:407] Connection header received
0619 02:53:59.223281 (+ 1876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:59.223287 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:59.223365 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:59.223462 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:59.224894 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.225646 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.227275 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.227789 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.227910 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:59.231261 (+ 3351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:59.231265 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:59.231268 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:59.231299 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:59.234220 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:59.234946 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:59.234950 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:59.234952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:59.235077 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:59.235482 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:59.235485 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:59.235487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:59.235739 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:59.235872 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:59.236540 (+ 668us) server_negotiation.cc:299] Negotiation successful
0619 02:53:59.236757 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":63}
I20260619 02:53:59.238039 2344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:59.220373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.2:33337)
0619 02:53:59.221230 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:59.221267 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:59.223022 (+ 1755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:59.223694 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:59.223702 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:59.224206 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:53:59.224729 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:59.224746 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.225786 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.225790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:59.227142 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:59.227154 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.228004 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.228010 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:59.228238 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:59.228867 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:53:59.228888 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:53:59.231068 (+ 2180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:53:59.234345 (+ 3277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:59.234355 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:59.234372 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:59.234829 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:59.235175 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:59.235181 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:59.235185 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:59.235362 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:59.235854 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:53:59.235862 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:53:59.236189 (+ 327us) client_negotiation.cc:769] Sending connection context
0619 02:53:59.236408 (+ 219us) client_negotiation.cc:240] Negotiation successful
0619 02:53:59.236907 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":277,"threads_started":1}
I20260619 02:53:59.239533 2342 heartbeater.cc:347] Connected to a master server at 127.18.134.62:35511
I20260619 02:53:59.239825 2342 heartbeater.cc:464] Registering TS with master...
I20260619 02:53:59.240530 2342 heartbeater.cc:511] Master 127.18.134.62:35511 requested a full tablet report, sending...
I20260619 02:53:59.242012 2019 ts_manager.cc:194] Registered new tserver with Master: 8451d39ceeb948e0814cbdc529dae4aa (127.18.134.2:37295)
I20260619 02:53:59.243063 2019 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.2@KRBTEST.COM'} at 127.18.134.2:33337
WARNING: no policy specified for kudu/127.18.134.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
I20260619 02:53:59.277137 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:0
--local_ip_for_outbound_sockets=127.18.134.3
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:35511
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:45187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:53:59.587844 2347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:53:59.588119 2347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:53:59.588212 2347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:53:59.595932 2347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:53:59.596040 2347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:53:59.596194 2347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:53:59.605854 2347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:45187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:35511
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:53:59.607125 2347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:53:59.608326 2347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:53:59.621064 2352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:59.620915 2353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:53:59.620959 2355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:53:59.622054 2347 server_base.cc:1034] running on GCE node
I20260619 02:53:59.622586 2347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:53:59.623594 2347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:53:59.624797 2347 hybrid_clock.cc:648] HybridClock initialized: now 1781837639624742 us; error 46 us; skew 500 ppm
I20260619 02:53:59.625089 2347 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:53:59 dist-test-slave-9mr0 krb5kdc[1974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837639, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:53:59.628137 2347 init.cc:377] Logged in from keytab as kudu/127.18.134.3@KRBTEST.COM (short username kudu)
I20260619 02:53:59.629503 2347 webserver.cc:460] Webserver started at http://127.18.134.3:39907/ using document root <none> and password file <none>
I20260619 02:53:59.630117 2347 fs_manager.cc:359] Metadata directory not provided
I20260619 02:53:59.630219 2347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:53:59.630511 2347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:53:59.633302 2347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/data/instance:
uuid: "cf9095bb0bd54375a245ae790a8ff346"
format_stamp: "Formatted at 2026-06-19 02:53:59 on dist-test-slave-9mr0"
server_key: "2f5a180c644def0f23a35e6e36963ac5"
server_key_iv: "dea10d9cbb32799a92d5653ca4024041"
server_key_version: "encryptionkey@0"
I20260619 02:53:59.634043 2347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance:
uuid: "cf9095bb0bd54375a245ae790a8ff346"
format_stamp: "Formatted at 2026-06-19 02:53:59 on dist-test-slave-9mr0"
server_key: "2f5a180c644def0f23a35e6e36963ac5"
server_key_iv: "dea10d9cbb32799a92d5653ca4024041"
server_key_version: "encryptionkey@0"
I20260619 02:53:59.639129 2347 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 02:53:59.642339 2362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:59.643610 2347 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:53:59.643756 2347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "cf9095bb0bd54375a245ae790a8ff346"
format_stamp: "Formatted at 2026-06-19 02:53:59 on dist-test-slave-9mr0"
server_key: "2f5a180c644def0f23a35e6e36963ac5"
server_key_iv: "dea10d9cbb32799a92d5653ca4024041"
server_key_version: "encryptionkey@0"
I20260619 02:53:59.643945 2347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:53:59.669773 2347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:53:59.673978 2347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:53:59.674238 2347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:53:59.675199 2347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:53:59.676504 2347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:53:59.676581 2347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:59.676707 2347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:53:59.676759 2347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:53:59.707175 2347 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:43255
I20260619 02:53:59.707218 2474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:43255 every 8 connection(s)
I20260619 02:53:59.708376 2347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:53:59.718475 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 2347
I20260619 02:53:59.718700 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance
I20260619 02:53:59.719264 18968 external_mini_cluster.cc:1428] Setting key 057032264e67c525098974441cbc10ef
Jun 19 02:53:59 dist-test-slave-9mr0 krb5kdc[1974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837639, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:53:59.727514 2210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:59.710963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:44433 (local address 127.18.134.62:35511)
0619 02:53:59.711168 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:59.711175 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:59.712329 (+ 1154us) server_negotiation.cc:407] Connection header received
0619 02:53:59.714043 (+ 1714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:59.714048 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:59.714121 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:59.714217 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:59.715604 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.716349 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.717843 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.718319 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.718437 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:59.722266 (+ 3829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:59.722273 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:59.722278 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:59.722314 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:59.724849 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:59.725543 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:59.725546 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:59.725548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:59.725657 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:59.726053 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:59.726056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:59.726058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:59.726334 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:59.726441 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:59.727120 (+ 679us) server_negotiation.cc:299] Negotiation successful
0619 02:53:59.727290 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:53:59.728551 2477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:59.711499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.3:44433)
0619 02:53:59.712168 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:59.712208 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:59.713809 (+ 1601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:59.714385 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:59.714392 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:59.714910 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:53:59.715422 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:59.715438 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.716495 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.716499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:59.717698 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:59.717713 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.718543 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.718550 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:59.718925 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:59.719702 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:53:59.719729 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:53:59.722029 (+ 2300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:53:59.724985 (+ 2956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:59.724994 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:59.725009 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:59.725449 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:59.725755 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:53:59.725758 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:53:59.725761 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:53:59.725958 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:53:59.726446 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:53:59.726452 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:53:59.726784 (+ 332us) client_negotiation.cc:769] Sending connection context
0619 02:53:59.727042 (+ 258us) client_negotiation.cc:240] Negotiation successful
0619 02:53:59.727416 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":268,"threads_started":1}
I20260619 02:53:59.729882 2475 heartbeater.cc:347] Connected to a master server at 127.18.134.62:35511
I20260619 02:53:59.730186 2475 heartbeater.cc:464] Registering TS with master...
I20260619 02:53:59.730895 2475 heartbeater.cc:511] Master 127.18.134.62:35511 requested a full tablet report, sending...
I20260619 02:53:59.732231 2019 ts_manager.cc:194] Registered new tserver with Master: cf9095bb0bd54375a245ae790a8ff346 (127.18.134.3:43255)
I20260619 02:53:59.733055 2019 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.3@KRBTEST.COM'} at 127.18.134.3:44433
I20260619 02:53:59.734829 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:53:59.750949 2210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:59.739265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59024 (local address 127.18.134.62:35511)
0619 02:53:59.739417 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:59.739421 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:59.739827 (+ 406us) server_negotiation.cc:407] Connection header received
0619 02:53:59.739960 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:59.739964 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:59.740021 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:59.740110 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:53:59.741434 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.742133 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.743587 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.743985 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.744074 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:59.746204 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:53:59.746208 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:53:59.746211 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:53:59.746235 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:53:59.748697 (+ 2462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:59.749330 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:59.749333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:59.749336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:59.749456 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:53:59.749731 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:53:59.749734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:53:59.749736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:53:59.750035 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:53:59.750221 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:59.750531 (+ 310us) server_negotiation.cc:299] Negotiation successful
0619 02:53:59.750729 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 02:53:59.751246 2208 heartbeater.cc:503] Master 127.18.134.62:35511 was elected leader, sending a full tablet report...
I20260619 02:53:59.754943 2019 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:53:59.761760 2019 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:53:59.783916 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:59.773441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:43255 (local address 127.0.0.1:60998)
0619 02:53:59.774277 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:59.774320 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:59.774529 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:59.775735 (+ 1206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:59.775742 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:59.775783 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:53:59.776153 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:59.776166 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.778442 (+ 2276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.778447 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:59.780924 (+ 2477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:59.780939 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.782855 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.782861 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:59.783061 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:59.783113 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 02:53:59.783225 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 02:53:59.783425 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":38,"thread_start_us":272,"threads_started":1}
I20260619 02:53:59.784488 2490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:59.775265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50926 (local address 127.18.134.1:41291)
0619 02:53:59.775904 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:59.775912 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:59.775939 (+ 27us) server_negotiation.cc:407] Connection header received
0619 02:53:59.776033 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:59.776040 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:59.776244 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:59.776417 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:53:59.777220 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.778787 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.781205 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.782687 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.782805 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:59.783053 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:59.784010 (+ 957us) server_negotiation.cc:299] Negotiation successful
0619 02:53:59.784171 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":311,"threads_started":1}
I20260619 02:53:59.784632 2489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:59.774678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:41291 (local address 127.0.0.1:50926)
0619 02:53:59.775169 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:59.775186 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:59.775291 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:59.776633 (+ 1342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:59.776637 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:59.776660 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:53:59.777033 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:59.777045 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.778955 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.778961 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:59.781052 (+ 2091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:59.781064 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.783221 (+ 2157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.783228 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:59.783445 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:59.783475 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 02:53:59.783599 (+ 124us) client_negotiation.cc:240] Negotiation successful
0619 02:53:59.783753 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":1,"thread_start_us":694,"threads_started":1}
I20260619 02:53:59.785645 2488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:59.773722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60998 (local address 127.18.134.3:43255)
0619 02:53:59.775182 (+ 1460us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:59.775189 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:59.775214 (+ 25us) server_negotiation.cc:407] Connection header received
0619 02:53:59.775286 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:59.775291 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:59.775492 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:59.776346 (+ 854us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:53:59.776634 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.778290 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.781140 (+ 2850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.782659 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.784831 (+ 2172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:59.785066 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:59.785141 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 02:53:59.785298 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1312,"thread_start_us":1171,"threads_started":1}
I20260619 02:53:59.788612 2143 tablet_service.cc:1467] Processing CreateTablet for tablet 40666e6bfeab45bd8e34a72554cdcdc1 (DEFAULT_TABLE table=test-table [id=5cf1e4f278d740f0b383e19c6e3dcc3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:53:59.788622 2410 tablet_service.cc:1467] Processing CreateTablet for tablet 40666e6bfeab45bd8e34a72554cdcdc1 (DEFAULT_TABLE table=test-table [id=5cf1e4f278d740f0b383e19c6e3dcc3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:53:59.790055 2410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40666e6bfeab45bd8e34a72554cdcdc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:53:59.790051 2143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40666e6bfeab45bd8e34a72554cdcdc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:53:59.795719 2492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:59.776155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42814 (local address 127.18.134.2:37295)
0619 02:53:59.785338 (+ 9183us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:59.785348 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:59.785374 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:53:59.785472 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:59.785479 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:59.785746 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:59.786470 (+ 724us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:53:59.789462 (+ 2992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.790985 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.793146 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.794573 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.794753 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:59.794958 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:59.795214 (+ 256us) server_negotiation.cc:299] Negotiation successful
0619 02:53:59.795386 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9027,"thread_start_us":8923,"threads_started":1}
I20260619 02:53:59.796479 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:59.775676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:37295 (local address 127.0.0.1:42814)
0619 02:53:59.784150 (+ 8474us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:59.784167 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:59.784274 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:59.786470 (+ 2196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:59.786474 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:59.786502 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:53:59.786696 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:59.786707 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.791169 (+ 4462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.791175 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:59.792970 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:59.792985 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.794773 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.794780 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:59.795013 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:59.795049 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 02:53:59.795158 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 02:53:59.795308 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8383,"thread_start_us":9626,"threads_started":1}
I20260619 02:53:59.800634 2277 tablet_service.cc:1467] Processing CreateTablet for tablet 40666e6bfeab45bd8e34a72554cdcdc1 (DEFAULT_TABLE table=test-table [id=5cf1e4f278d740f0b383e19c6e3dcc3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:53:59.802098 2277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40666e6bfeab45bd8e34a72554cdcdc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:53:59.802189 2493 tablet_bootstrap.cc:492] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07: Bootstrap starting.
I20260619 02:53:59.805855 2493 tablet_bootstrap.cc:654] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07: Neither blocks nor log segments found. Creating new log.
I20260619 02:53:59.806777 2494 tablet_bootstrap.cc:492] T 40666e6bfeab45bd8e34a72554cdcdc1 P cf9095bb0bd54375a245ae790a8ff346: Bootstrap starting.
I20260619 02:53:59.807158 2493 log.cc:824] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07: Log is configured to *not* fsync() on all Append() calls
I20260619 02:53:59.809937 2494 tablet_bootstrap.cc:654] T 40666e6bfeab45bd8e34a72554cdcdc1 P cf9095bb0bd54375a245ae790a8ff346: Neither blocks nor log segments found. Creating new log.
I20260619 02:53:59.810416 2493 tablet_bootstrap.cc:492] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07: No bootstrap required, opened a new log
I20260619 02:53:59.810721 2493 ts_tablet_manager.cc:1397] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260619 02:53:59.811015 2494 log.cc:824] T 40666e6bfeab45bd8e34a72554cdcdc1 P cf9095bb0bd54375a245ae790a8ff346: Log is configured to *not* fsync() on all Append() calls
I20260619 02:53:59.814134 2494 tablet_bootstrap.cc:492] T 40666e6bfeab45bd8e34a72554cdcdc1 P cf9095bb0bd54375a245ae790a8ff346: No bootstrap required, opened a new log
I20260619 02:53:59.814375 2494 ts_tablet_manager.cc:1397] T 40666e6bfeab45bd8e34a72554cdcdc1 P cf9095bb0bd54375a245ae790a8ff346: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20260619 02:53:59.817487 2497 tablet_bootstrap.cc:492] T 40666e6bfeab45bd8e34a72554cdcdc1 P 8451d39ceeb948e0814cbdc529dae4aa: Bootstrap starting.
I20260619 02:53:59.820204 2493 raft_consensus.cc:348] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9095bb0bd54375a245ae790a8ff346" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43255 } } peers { permanent_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 41291 } } peers { permanent_uuid: "8451d39ceeb948e0814cbdc529dae4aa" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37295 } }
I20260619 02:53:59.820544 2493 raft_consensus.cc:374] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:53:59.820654 2493 raft_consensus.cc:729] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2da3d0dd4eb1432787df0d9838ba8b07, State: Initialized, Role: FOLLOWER
I20260619 02:53:59.821192 2497 tablet_bootstrap.cc:654] T 40666e6bfeab45bd8e34a72554cdcdc1 P 8451d39ceeb948e0814cbdc529dae4aa: Neither blocks nor log segments found. Creating new log.
I20260619 02:53:59.821193 2493 consensus_queue.cc:260] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9095bb0bd54375a245ae790a8ff346" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43255 } } peers { permanent_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 41291 } } peers { permanent_uuid: "8451d39ceeb948e0814cbdc529dae4aa" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37295 } }
I20260619 02:53:59.822873 2493 ts_tablet_manager.cc:1428] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260619 02:53:59.823175 2497 log.cc:824] T 40666e6bfeab45bd8e34a72554cdcdc1 P 8451d39ceeb948e0814cbdc529dae4aa: Log is configured to *not* fsync() on all Append() calls
I20260619 02:53:59.824070 2494 raft_consensus.cc:348] T 40666e6bfeab45bd8e34a72554cdcdc1 P cf9095bb0bd54375a245ae790a8ff346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9095bb0bd54375a245ae790a8ff346" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43255 } } peers { permanent_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 41291 } } peers { permanent_uuid: "8451d39ceeb948e0814cbdc529dae4aa" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37295 } }
I20260619 02:53:59.824400 2494 raft_consensus.cc:374] T 40666e6bfeab45bd8e34a72554cdcdc1 P cf9095bb0bd54375a245ae790a8ff346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:53:59.824493 2494 raft_consensus.cc:729] T 40666e6bfeab45bd8e34a72554cdcdc1 P cf9095bb0bd54375a245ae790a8ff346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf9095bb0bd54375a245ae790a8ff346, State: Initialized, Role: FOLLOWER
I20260619 02:53:59.825019 2494 consensus_queue.cc:260] T 40666e6bfeab45bd8e34a72554cdcdc1 P cf9095bb0bd54375a245ae790a8ff346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9095bb0bd54375a245ae790a8ff346" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43255 } } peers { permanent_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 41291 } } peers { permanent_uuid: "8451d39ceeb948e0814cbdc529dae4aa" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37295 } }
I20260619 02:53:59.826004 2475 heartbeater.cc:503] Master 127.18.134.62:35511 was elected leader, sending a full tablet report...
I20260619 02:53:59.826340 2497 tablet_bootstrap.cc:492] T 40666e6bfeab45bd8e34a72554cdcdc1 P 8451d39ceeb948e0814cbdc529dae4aa: No bootstrap required, opened a new log
I20260619 02:53:59.826588 2497 ts_tablet_manager.cc:1397] T 40666e6bfeab45bd8e34a72554cdcdc1 P 8451d39ceeb948e0814cbdc529dae4aa: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260619 02:53:59.826609 2494 ts_tablet_manager.cc:1428] T 40666e6bfeab45bd8e34a72554cdcdc1 P cf9095bb0bd54375a245ae790a8ff346: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260619 02:53:59.834396 2497 raft_consensus.cc:348] T 40666e6bfeab45bd8e34a72554cdcdc1 P 8451d39ceeb948e0814cbdc529dae4aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9095bb0bd54375a245ae790a8ff346" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43255 } } peers { permanent_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 41291 } } peers { permanent_uuid: "8451d39ceeb948e0814cbdc529dae4aa" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37295 } }
I20260619 02:53:59.834780 2497 raft_consensus.cc:374] T 40666e6bfeab45bd8e34a72554cdcdc1 P 8451d39ceeb948e0814cbdc529dae4aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:53:59.834874 2497 raft_consensus.cc:729] T 40666e6bfeab45bd8e34a72554cdcdc1 P 8451d39ceeb948e0814cbdc529dae4aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8451d39ceeb948e0814cbdc529dae4aa, State: Initialized, Role: FOLLOWER
I20260619 02:53:59.835390 2497 consensus_queue.cc:260] T 40666e6bfeab45bd8e34a72554cdcdc1 P 8451d39ceeb948e0814cbdc529dae4aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9095bb0bd54375a245ae790a8ff346" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43255 } } peers { permanent_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 41291 } } peers { permanent_uuid: "8451d39ceeb948e0814cbdc529dae4aa" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37295 } }
I20260619 02:53:59.836418 2342 heartbeater.cc:503] Master 127.18.134.62:35511 was elected leader, sending a full tablet report...
I20260619 02:53:59.837078 2497 ts_tablet_manager.cc:1428] T 40666e6bfeab45bd8e34a72554cdcdc1 P 8451d39ceeb948e0814cbdc529dae4aa: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260619 02:53:59.913793 2498 raft_consensus.cc:482] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:53:59.914121 2498 raft_consensus.cc:504] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9095bb0bd54375a245ae790a8ff346" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43255 } } peers { permanent_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 41291 } } peers { permanent_uuid: "8451d39ceeb948e0814cbdc529dae4aa" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37295 } }
I20260619 02:53:59.915859 2498 leader_election.cc:288] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf9095bb0bd54375a245ae790a8ff346 (127.18.134.3:43255), 8451d39ceeb948e0814cbdc529dae4aa (127.18.134.2:37295)
I20260619 02:53:59.922144 2488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:59.916270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:37155 (local address 127.18.134.3:43255)
0619 02:53:59.916451 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:59.916456 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:59.916966 (+ 510us) server_negotiation.cc:407] Connection header received
0619 02:53:59.917241 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:59.917249 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:59.917330 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:59.917388 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:53:59.918043 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.918810 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.920526 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.921277 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.921421 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:59.921496 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:59.921779 (+ 283us) server_negotiation.cc:299] Negotiation successful
0619 02:53:59.921905 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 02:53:59.922291 2502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:59.916242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:43255 (local address 127.18.134.1:37155)
0619 02:53:59.916863 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:59.916882 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:59.917021 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:59.917585 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:59.917590 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:59.917627 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:53:59.917904 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:59.917911 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.918932 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.918937 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:59.920376 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:59.920388 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.921476 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.921482 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:59.921621 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:59.921646 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 02:53:59.921726 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 02:53:59.921860 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"mutex_wait_us":23,"thread_start_us":342,"threads_started":1}
I20260619 02:53:59.922446 2492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:59.917165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:35075 (local address 127.18.134.2:37295)
0619 02:53:59.917366 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:59.917372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:59.917696 (+ 324us) server_negotiation.cc:407] Connection header received
0619 02:53:59.917890 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:59.917893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:59.917953 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:59.918025 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:53:59.918509 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.919240 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.920872 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.921597 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.921730 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:59.921800 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:59.922077 (+ 277us) server_negotiation.cc:299] Negotiation successful
0619 02:53:59.922201 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 02:53:59.922595 2503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:59.917021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:37295 (local address 127.18.134.1:35075)
0619 02:53:59.917569 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0619 02:53:59.917582 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:53:59.917749 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:53:59.918053 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:53:59.918057 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:53:59.918080 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:53:59.918364 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:59.918372 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.919370 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.919374 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:59.920726 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:53:59.920737 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.921802 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.921808 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:53:59.921937 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:59.921960 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 02:53:59.922038 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 02:53:59.922161 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":1,"thread_start_us":254,"threads_started":1}
I20260619 02:53:59.923535 2430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40666e6bfeab45bd8e34a72554cdcdc1" candidate_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf9095bb0bd54375a245ae790a8ff346" is_pre_election: true
I20260619 02:53:59.923799 2297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40666e6bfeab45bd8e34a72554cdcdc1" candidate_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8451d39ceeb948e0814cbdc529dae4aa" is_pre_election: true
I20260619 02:53:59.924014 2430 raft_consensus.cc:2454] T 40666e6bfeab45bd8e34a72554cdcdc1 P cf9095bb0bd54375a245ae790a8ff346 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2da3d0dd4eb1432787df0d9838ba8b07 in term 0.
I20260619 02:53:59.924234 2297 raft_consensus.cc:2454] T 40666e6bfeab45bd8e34a72554cdcdc1 P 8451d39ceeb948e0814cbdc529dae4aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2da3d0dd4eb1432787df0d9838ba8b07 in term 0.
I20260619 02:53:59.924674 2096 leader_election.cc:302] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2da3d0dd4eb1432787df0d9838ba8b07, cf9095bb0bd54375a245ae790a8ff346; no voters:
I20260619 02:53:59.925071 2498 raft_consensus.cc:2781] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:53:59.925230 2498 raft_consensus.cc:482] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:53:59.925335 2498 raft_consensus.cc:3037] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:53:59.927569 2498 raft_consensus.cc:504] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9095bb0bd54375a245ae790a8ff346" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43255 } } peers { permanent_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 41291 } } peers { permanent_uuid: "8451d39ceeb948e0814cbdc529dae4aa" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37295 } }
I20260619 02:53:59.928154 2498 leader_election.cc:288] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07 [CANDIDATE]: Term 1 election: Requested vote from peers cf9095bb0bd54375a245ae790a8ff346 (127.18.134.3:43255), 8451d39ceeb948e0814cbdc529dae4aa (127.18.134.2:37295)
I20260619 02:53:59.928632 2430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40666e6bfeab45bd8e34a72554cdcdc1" candidate_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf9095bb0bd54375a245ae790a8ff346"
I20260619 02:53:59.928706 2297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40666e6bfeab45bd8e34a72554cdcdc1" candidate_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8451d39ceeb948e0814cbdc529dae4aa"
I20260619 02:53:59.928824 2430 raft_consensus.cc:3037] T 40666e6bfeab45bd8e34a72554cdcdc1 P cf9095bb0bd54375a245ae790a8ff346 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:53:59.928884 2297 raft_consensus.cc:3037] T 40666e6bfeab45bd8e34a72554cdcdc1 P 8451d39ceeb948e0814cbdc529dae4aa [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:53:59.931110 2430 raft_consensus.cc:2454] T 40666e6bfeab45bd8e34a72554cdcdc1 P cf9095bb0bd54375a245ae790a8ff346 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2da3d0dd4eb1432787df0d9838ba8b07 in term 1.
I20260619 02:53:59.931115 2297 raft_consensus.cc:2454] T 40666e6bfeab45bd8e34a72554cdcdc1 P 8451d39ceeb948e0814cbdc529dae4aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2da3d0dd4eb1432787df0d9838ba8b07 in term 1.
I20260619 02:53:59.931489 2096 leader_election.cc:302] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2da3d0dd4eb1432787df0d9838ba8b07, cf9095bb0bd54375a245ae790a8ff346; no voters:
I20260619 02:53:59.931787 2498 raft_consensus.cc:2781] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:53:59.932257 2498 raft_consensus.cc:686] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07 [term 1 LEADER]: Becoming Leader. State: Replica: 2da3d0dd4eb1432787df0d9838ba8b07, State: Running, Role: LEADER
I20260619 02:53:59.932807 2498 consensus_queue.cc:237] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9095bb0bd54375a245ae790a8ff346" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43255 } } peers { permanent_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 41291 } } peers { permanent_uuid: "8451d39ceeb948e0814cbdc529dae4aa" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37295 } }
I20260619 02:53:59.938305 2018 catalog_manager.cc:5495] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2da3d0dd4eb1432787df0d9838ba8b07 (127.18.134.1). New cstate: current_term: 1 leader_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf9095bb0bd54375a245ae790a8ff346" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 41291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8451d39ceeb948e0814cbdc529dae4aa" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37295 } health_report { overall_health: UNKNOWN } } }
W20260619 02:53:59.959882 2476 tablet.cc:2369] T 40666e6bfeab45bd8e34a72554cdcdc1 P cf9095bb0bd54375a245ae790a8ff346: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:53:59.969367 2343 tablet.cc:2369] T 40666e6bfeab45bd8e34a72554cdcdc1 P 8451d39ceeb948e0814cbdc529dae4aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:53:59.973325 2209 tablet.cc:2369] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:53:59.975433 2490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:53:59.969703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50930 (local address 127.18.134.1:41291)
0619 02:53:59.969873 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 02:53:59.969878 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:53:59.969900 (+ 22us) server_negotiation.cc:407] Connection header received
0619 02:53:59.970175 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:53:59.970178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:53:59.970252 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:53:59.970370 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:53:59.970817 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.971582 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.973074 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:53:59.973699 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:53:59.973823 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:53:59.974362 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:53:59.974900 (+ 538us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:53:59.975014 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 02:53:59.975129 (+ 115us) server_negotiation.cc:299] Negotiation successful
0619 02:53:59.975198 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 02:53:59.983238 2297 raft_consensus.cc:1261] T 40666e6bfeab45bd8e34a72554cdcdc1 P 8451d39ceeb948e0814cbdc529dae4aa [term 1 FOLLOWER]: Refusing update from remote peer 2da3d0dd4eb1432787df0d9838ba8b07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:53:59.983328 2430 raft_consensus.cc:1261] T 40666e6bfeab45bd8e34a72554cdcdc1 P cf9095bb0bd54375a245ae790a8ff346 [term 1 FOLLOWER]: Refusing update from remote peer 2da3d0dd4eb1432787df0d9838ba8b07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:53:59.984144 2504 consensus_queue.cc:1035] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8451d39ceeb948e0814cbdc529dae4aa" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:53:59.984519 2498 consensus_queue.cc:1035] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf9095bb0bd54375a245ae790a8ff346" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:53:59.994800 2507 mvcc.cc:205] Tried to move back new op lower bound from 7298406973361696768 to 7298406973169410048. Current Snapshot: MvccSnapshot[applied={T|T < 7298406973361696768}]
I20260619 02:54:00.009073 2019 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:54:00.009334 2019 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:54:00.012809 2019 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3: Sending DeleteTablet for 3 replicas of tablet 40666e6bfeab45bd8e34a72554cdcdc1
I20260619 02:54:00.013980 2410 tablet_service.cc:1514] Processing DeleteTablet for tablet 40666e6bfeab45bd8e34a72554cdcdc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:54:00 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:60998
I20260619 02:54:00.014145 2143 tablet_service.cc:1514] Processing DeleteTablet for tablet 40666e6bfeab45bd8e34a72554cdcdc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:54:00 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:50926
I20260619 02:54:00.014356 2277 tablet_service.cc:1514] Processing DeleteTablet for tablet 40666e6bfeab45bd8e34a72554cdcdc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:54:00 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:42814
I20260619 02:54:00.015388 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 2080
I20260619 02:54:00.024842 2519 tablet_replica.cc:330] T 40666e6bfeab45bd8e34a72554cdcdc1 P 8451d39ceeb948e0814cbdc529dae4aa: stopping tablet replica
I20260619 02:54:00.025229 2519 raft_consensus.cc:2229] T 40666e6bfeab45bd8e34a72554cdcdc1 P 8451d39ceeb948e0814cbdc529dae4aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:54:00.025671 2519 raft_consensus.cc:2258] T 40666e6bfeab45bd8e34a72554cdcdc1 P 8451d39ceeb948e0814cbdc529dae4aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:54:00.030155 2519 ts_tablet_manager.cc:1905] T 40666e6bfeab45bd8e34a72554cdcdc1 P 8451d39ceeb948e0814cbdc529dae4aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:54:00.037405 2517 tablet_replica.cc:330] T 40666e6bfeab45bd8e34a72554cdcdc1 P cf9095bb0bd54375a245ae790a8ff346: stopping tablet replica
I20260619 02:54:00.037824 2517 raft_consensus.cc:2229] T 40666e6bfeab45bd8e34a72554cdcdc1 P cf9095bb0bd54375a245ae790a8ff346 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:54:00.038352 2517 raft_consensus.cc:2258] T 40666e6bfeab45bd8e34a72554cdcdc1 P cf9095bb0bd54375a245ae790a8ff346 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:54:00.039211 2518 tablet_replica.cc:330] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07: stopping tablet replica
I20260619 02:54:00.039649 2518 raft_consensus.cc:2229] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:54:00.040364 2518 raft_consensus.cc:2258] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07 [term 1 FOLLOWER]: Raft consensus is shut down!
Jun 19 02:54:00 dist-test-slave-9mr0 krb5kdc[1974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.1@KRBTEST.COM
I20260619 02:54:00.042680 2517 ts_tablet_manager.cc:1905] T 40666e6bfeab45bd8e34a72554cdcdc1 P cf9095bb0bd54375a245ae790a8ff346: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:54:00.042702 2518 ts_tablet_manager.cc:1905] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:54:00.050150 2518 ts_tablet_manager.cc:1918] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:54:00.050452 2519 ts_tablet_manager.cc:1918] T 40666e6bfeab45bd8e34a72554cdcdc1 P 8451d39ceeb948e0814cbdc529dae4aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:54:00.050607 2519 log.cc:1196] T 40666e6bfeab45bd8e34a72554cdcdc1 P 8451d39ceeb948e0814cbdc529dae4aa: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/wal/wals/40666e6bfeab45bd8e34a72554cdcdc1
I20260619 02:54:00.050846 2490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:00.016459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50942 (local address 127.18.134.1:41291)
0619 02:54:00.016645 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 02:54:00.016651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:54:00.026464 (+ 9813us) server_negotiation.cc:407] Connection header received
0619 02:54:00.026658 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:54:00.026664 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:54:00.026773 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:54:00.026871 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:54:00.028955 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:00.030128 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:00.032319 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:00.033054 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:00.033150 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:54:00.042864 (+ 9714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:54:00.042877 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:54:00.042894 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:54:00.042944 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:54:00.046981 (+ 4037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:54:00.047966 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:54:00.047978 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:54:00.047982 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:54:00.048218 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:54:00.048679 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:54:00.048684 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:54:00.048687 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:54:00.049142 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:54:00.049428 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0619 02:54:00.050007 (+ 579us) server_negotiation.cc:299] Negotiation successful
0619 02:54:00.050248 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 02:54:00.051146 2519 ts_tablet_manager.cc:1939] T 40666e6bfeab45bd8e34a72554cdcdc1 P 8451d39ceeb948e0814cbdc529dae4aa: Deleting consensus metadata
I20260619 02:54:00.050401 2518 log.cc:1196] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/wal/wals/40666e6bfeab45bd8e34a72554cdcdc1
I20260619 02:54:00.051915 2203 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:54:00.053104 2003 catalog_manager.cc:4841] TS 8451d39ceeb948e0814cbdc529dae4aa (127.18.134.2:37295): tablet 40666e6bfeab45bd8e34a72554cdcdc1 (table test-table [id=5cf1e4f278d740f0b383e19c6e3dcc3f]) successfully deleted
I20260619 02:54:00.054073 2517 ts_tablet_manager.cc:1918] T 40666e6bfeab45bd8e34a72554cdcdc1 P cf9095bb0bd54375a245ae790a8ff346: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:54:00.054222 2517 log.cc:1196] T 40666e6bfeab45bd8e34a72554cdcdc1 P cf9095bb0bd54375a245ae790a8ff346: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/wal/wals/40666e6bfeab45bd8e34a72554cdcdc1
I20260619 02:54:00.054719 2517 ts_tablet_manager.cc:1939] T 40666e6bfeab45bd8e34a72554cdcdc1 P cf9095bb0bd54375a245ae790a8ff346: Deleting consensus metadata
I20260619 02:54:00.056397 2003 catalog_manager.cc:4841] TS cf9095bb0bd54375a245ae790a8ff346 (127.18.134.3:43255): tablet 40666e6bfeab45bd8e34a72554cdcdc1 (table test-table [id=5cf1e4f278d740f0b383e19c6e3dcc3f]) successfully deleted
I20260619 02:54:00.149516 2518 ts_tablet_manager.cc:1939] T 40666e6bfeab45bd8e34a72554cdcdc1 P 2da3d0dd4eb1432787df0d9838ba8b07: Deleting consensus metadata
I20260619 02:54:00.150246 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 2080
I20260619 02:54:00.165253 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 2214
W20260619 02:54:00.165477 2003 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.134.1:41291 (error 108)
I20260619 02:54:00.167300 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:00.166910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:41291 (local address 127.0.0.1:50954)
0619 02:54:00.167058 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:00.167151 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:41291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260619 02:54:00.167500 2003 catalog_manager.cc:4584] TS 2da3d0dd4eb1432787df0d9838ba8b07 (127.18.134.1:41291): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 40666e6bfeab45bd8e34a72554cdcdc1: Network error: Client connection negotiation failed: client connection to 127.18.134.1:41291: connect: Connection refused (error 111)
Jun 19 02:54:00 dist-test-slave-9mr0 krb5kdc[1974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.2@KRBTEST.COM
I20260619 02:54:00.179788 2492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:00.165945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42826 (local address 127.18.134.2:37295)
0619 02:54:00.166148 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0619 02:54:00.166154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:54:00.166367 (+ 213us) server_negotiation.cc:407] Connection header received
0619 02:54:00.166547 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:54:00.166552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:54:00.166660 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:54:00.166779 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:54:00.168253 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:00.168966 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:00.170209 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:00.170563 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:00.170694 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:54:00.174561 (+ 3867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:54:00.174573 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:54:00.174586 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:54:00.174656 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:54:00.177295 (+ 2639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:54:00.177933 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:54:00.177939 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:54:00.177942 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:54:00.178050 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:54:00.178358 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:54:00.178360 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:54:00.178362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:54:00.178834 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:54:00.179065 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0619 02:54:00.179338 (+ 273us) server_negotiation.cc:299] Negotiation successful
0619 02:54:00.179493 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 02:54:00.180433 2337 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:54:00.221230 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:00.220775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:41291 (local address 127.0.0.1:50956)
0619 02:54:00.220987 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:00.221088 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:41291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260619 02:54:00.222200 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:00.221831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:41291 (local address 127.0.0.1:50968)
0619 02:54:00.221971 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:00.222062 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:41291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 02:54:00.267760 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 2214
I20260619 02:54:00.280905 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 2347
I20260619 02:54:00.282616 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:00.282202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:41291 (local address 127.0.0.1:50982)
0619 02:54:00.282362 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:00.282440 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:41291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260619 02:54:00.283618 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:00.283253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:41291 (local address 127.0.0.1:50998)
0619 02:54:00.283390 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:00.283470 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:41291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
Jun 19 02:54:00 dist-test-slave-9mr0 krb5kdc[1974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.3@KRBTEST.COM
I20260619 02:54:00.296183 2488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:00.281625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32768 (local address 127.18.134.3:43255)
0619 02:54:00.281822 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 02:54:00.281828 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:54:00.281849 (+ 21us) server_negotiation.cc:407] Connection header received
0619 02:54:00.282064 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:54:00.282070 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:54:00.282157 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:54:00.282240 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:54:00.283680 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:00.284481 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:00.285925 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:00.286316 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:00.286430 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:54:00.290568 (+ 4138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:54:00.290583 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:54:00.290600 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:54:00.290655 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:54:00.293306 (+ 2651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:54:00.293974 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:54:00.293984 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:54:00.293987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:54:00.294148 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:54:00.294508 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:54:00.294513 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:54:00.294517 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:54:00.295030 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:54:00.295298 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0619 02:54:00.295639 (+ 341us) server_negotiation.cc:299] Negotiation successful
0619 02:54:00.295871 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 02:54:00.296816 2470 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:54:00.364537 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:00.364001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:41291 (local address 127.0.0.1:51002)
0619 02:54:00.364263 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:00.364361 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:41291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260619 02:54:00.365628 2487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:00.365232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:41291 (local address 127.0.0.1:51018)
0619 02:54:00.365432 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:00.365505 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:41291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260619 02:54:00.384620 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 2347
I20260619 02:54:00.398590 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 1987
I20260619 02:54:00.399734 2049 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:54:00.507961 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 1987
I20260619 02:54:20.525315 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:35511
--webserver_interface=127.18.134.62
--webserver_port=45413
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:45187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:35511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:54:20.845189 2526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:54:20.845475 2526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:54:20.845582 2526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:54:20.852756 2526 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 02:54:20.852869 2526 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 02:54:20.852934 2526 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 02:54:20.853001 2526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:54:20.853076 2526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:54:20.853170 2526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:54:20.853235 2526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:54:20.853310 2526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:54:20.863243 2526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:45187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:35511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:35511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=45413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:54:20.864506 2526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:54:20.865823 2526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:54:20.887797 2532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:54:20.887778 2533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:54:20.887820 2535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:54:20.893594 2526 server_base.cc:1034] running on GCE node
I20260619 02:54:20.894515 2526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:54:20.896219 2526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:54:20.897496 2526 hybrid_clock.cc:648] HybridClock initialized: now 1781837660897415 us; error 67 us; skew 500 ppm
I20260619 02:54:20.897894 2526 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:54:20 dist-test-slave-9mr0 krb5kdc[1974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837660, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:54:20.906404 2526 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:54:20.908320 2526 webserver.cc:460] Webserver started at http://127.18.134.62:45413/ using document root <none> and password file <none>
I20260619 02:54:20.909132 2526 fs_manager.cc:359] Metadata directory not provided
I20260619 02:54:20.909245 2526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:54:20.915608 2526 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 02:54:20.918912 2542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:54:20.920342 2526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 02:54:20.920519 2526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "2d04b4e2748641509554c98f7efecfc3"
format_stamp: "Formatted at 2026-06-19 02:53:58 on dist-test-slave-9mr0"
server_key: "c5584d847587e46c08071fd2eb906d26"
server_key_iv: "43001b69266fabf6ed8d4ca2444ee469"
server_key_version: "encryptionkey@0"
I20260619 02:54:20.921567 2526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:54:20.940310 2526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:54:20.944757 2526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:54:20.945037 2526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:54:20.963189 2526 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:35511
I20260619 02:54:20.963227 2593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:35511 every 8 connection(s)
I20260619 02:54:20.964668 2526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:54:20.968302 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 2526
I20260619 02:54:20.969262 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:41291
--local_ip_for_outbound_sockets=127.18.134.1
--tserver_master_addrs=127.18.134.62:35511
--webserver_port=38429
--webserver_interface=127.18.134.1
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:45187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:54:20.973157 2594 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:54:20.977842 2594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3: Bootstrap starting.
I20260619 02:54:20.997668 2594 log.cc:824] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3: Log is configured to *not* fsync() on all Append() calls
I20260619 02:54:21.009774 2594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:54:21.010277 2594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3: Bootstrap complete.
I20260619 02:54:21.019335 2594 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d04b4e2748641509554c98f7efecfc3" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35511 } }
I20260619 02:54:21.020231 2594 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d04b4e2748641509554c98f7efecfc3, State: Initialized, Role: FOLLOWER
I20260619 02:54:21.020860 2594 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d04b4e2748641509554c98f7efecfc3" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35511 } }
I20260619 02:54:21.021088 2594 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:54:21.021214 2594 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:54:21.021378 2594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [term 1 FOLLOWER]: Advancing to term 2
I20260619 02:54:21.025527 2594 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d04b4e2748641509554c98f7efecfc3" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35511 } }
I20260619 02:54:21.026070 2594 leader_election.cc:302] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d04b4e2748641509554c98f7efecfc3; no voters:
I20260619 02:54:21.026731 2594 leader_election.cc:288] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 02:54:21.026871 2598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 02:54:21.028928 2594 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:54:21.033990 2598 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [term 2 LEADER]: Becoming Leader. State: Replica: 2d04b4e2748641509554c98f7efecfc3, State: Running, Role: LEADER
I20260619 02:54:21.034657 2598 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d04b4e2748641509554c98f7efecfc3" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35511 } }
I20260619 02:54:21.037292 2599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2d04b4e2748641509554c98f7efecfc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d04b4e2748641509554c98f7efecfc3" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35511 } } }
I20260619 02:54:21.037709 2599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [sys.catalog]: This master's current role is: LEADER
I20260619 02:54:21.038259 2600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d04b4e2748641509554c98f7efecfc3. Latest consensus state: current_term: 2 leader_uuid: "2d04b4e2748641509554c98f7efecfc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d04b4e2748641509554c98f7efecfc3" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35511 } } }
I20260619 02:54:21.038525 2600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [sys.catalog]: This master's current role is: LEADER
I20260619 02:54:21.045277 2609 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:54:21.053586 2609 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:54:21.055043 2609 catalog_manager.cc:1259] Loaded cluster ID: 070be589d2284f839da167d7b368c425
I20260619 02:54:21.055193 2609 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:54:21.059034 2609 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:54:21.060556 2609 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3: Loaded TSK: 0
I20260619 02:54:21.106284 2609 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3: Generated new TSK 1
I20260619 02:54:21.106772 2609 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 02:54:21.314195 2596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:54:21.314462 2596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:54:21.314574 2596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:54:21.321805 2596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:54:21.321918 2596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:54:21.322057 2596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:54:21.332078 2596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:45187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:41291
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=38429
--tserver_master_addrs=127.18.134.62:35511
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:54:21.333312 2596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:54:21.334555 2596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:54:21.349004 2625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:54:21.348974 2623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:54:21.349707 2622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:54:21.349946 2596 server_base.cc:1034] running on GCE node
I20260619 02:54:21.350533 2596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:54:21.351498 2596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:54:21.352697 2596 hybrid_clock.cc:648] HybridClock initialized: now 1781837661352636 us; error 58 us; skew 500 ppm
I20260619 02:54:21.352986 2596 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:54:21 dist-test-slave-9mr0 krb5kdc[1974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837661, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:54:21.356016 2596 init.cc:377] Logged in from keytab as kudu/127.18.134.1@KRBTEST.COM (short username kudu)
I20260619 02:54:21.357494 2596 webserver.cc:460] Webserver started at http://127.18.134.1:38429/ using document root <none> and password file <none>
I20260619 02:54:21.358157 2596 fs_manager.cc:359] Metadata directory not provided
I20260619 02:54:21.358281 2596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:54:21.364068 2596 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 02:54:21.366917 2632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:54:21.368355 2596 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:54:21.368510 2596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "2da3d0dd4eb1432787df0d9838ba8b07"
format_stamp: "Formatted at 2026-06-19 02:53:58 on dist-test-slave-9mr0"
server_key: "3ba52d3a1c4e3e634d225f1a0b82e3ff"
server_key_iv: "0d4817d0a8d4f845f93f57ad7aaa1564"
server_key_version: "encryptionkey@0"
I20260619 02:54:21.369467 2596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:54:21.385599 2596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:54:21.389755 2596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:54:21.390020 2596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:54:21.391006 2596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:54:21.392295 2596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:54:21.392388 2596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:54:21.392494 2596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:54:21.392567 2596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:54:21.424149 2596 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:41291
I20260619 02:54:21.424194 2744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:41291 every 8 connection(s)
I20260619 02:54:21.425647 2596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:54:21.434435 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 2596
I20260619 02:54:21.435101 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:37295
--local_ip_for_outbound_sockets=127.18.134.2
--tserver_master_addrs=127.18.134.62:35511
--webserver_port=44289
--webserver_interface=127.18.134.2
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:45187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:54:21 dist-test-slave-9mr0 krb5kdc[1974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837661, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:54:21.450204 2747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:21.428365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:46541 (local address 127.18.134.62:35511)
0619 02:54:21.428899 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0619 02:54:21.428911 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:54:21.429565 (+ 654us) server_negotiation.cc:407] Connection header received
0619 02:54:21.431657 (+ 2092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:54:21.431669 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:54:21.432005 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:54:21.432304 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:54:21.433655 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:21.435001 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:21.437344 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:21.437975 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:21.438108 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:54:21.442191 (+ 4083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:54:21.442199 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:54:21.442214 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:54:21.442258 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:54:21.445748 (+ 3490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:54:21.446470 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:54:21.446478 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:54:21.446484 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:54:21.446707 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:54:21.447277 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:54:21.447282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:54:21.447286 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:54:21.447944 (+ 658us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:54:21.448093 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 02:54:21.448969 (+ 876us) server_negotiation.cc:299] Negotiation successful
0619 02:54:21.449282 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":212,"threads_started":1}
I20260619 02:54:21.450464 2748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:21.428627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.1:46541)
0619 02:54:21.429405 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:21.429449 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:54:21.431175 (+ 1726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:54:21.432420 (+ 1245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:54:21.432427 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:54:21.432948 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:54:21.433462 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:54:21.433478 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:54:21.435174 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:54:21.435180 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:54:21.437196 (+ 2016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:54:21.437211 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:54:21.438201 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:54:21.438208 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:54:21.438444 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:54:21.439148 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:54:21.439175 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:54:21.441854 (+ 2679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:54:21.445910 (+ 4056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:54:21.445917 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:54:21.445930 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:54:21.446347 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:54:21.446836 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:54:21.446840 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:54:21.446844 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:54:21.447125 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:54:21.448094 (+ 969us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:54:21.448102 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:54:21.448518 (+ 416us) client_negotiation.cc:769] Sending connection context
0619 02:54:21.448794 (+ 276us) client_negotiation.cc:240] Negotiation successful
0619 02:54:21.449314 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":299,"threads_started":1}
I20260619 02:54:21.453886 2745 heartbeater.cc:347] Connected to a master server at 127.18.134.62:35511
I20260619 02:54:21.454221 2745 heartbeater.cc:464] Registering TS with master...
I20260619 02:54:21.455027 2745 heartbeater.cc:511] Master 127.18.134.62:35511 requested a full tablet report, sending...
I20260619 02:54:21.457571 2559 ts_manager.cc:194] Registered new tserver with Master: 2da3d0dd4eb1432787df0d9838ba8b07 (127.18.134.1:41291)
I20260619 02:54:21.459743 2559 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.1@KRBTEST.COM'} at 127.18.134.1:46541
W20260619 02:54:21.756057 2749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:54:21.756332 2749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:54:21.756425 2749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:54:21.763623 2749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:54:21.763737 2749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:54:21.763859 2749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:54:21.773469 2749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:45187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:37295
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=44289
--tserver_master_addrs=127.18.134.62:35511
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:54:21.774777 2749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:54:21.776053 2749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:54:21.788841 2755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:54:21.788800 2754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:54:21.789534 2757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:54:21.789723 2749 server_base.cc:1034] running on GCE node
I20260619 02:54:21.790380 2749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:54:21.791359 2749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:54:21.792565 2749 hybrid_clock.cc:648] HybridClock initialized: now 1781837661792513 us; error 48 us; skew 500 ppm
I20260619 02:54:21.792896 2749 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:54:21 dist-test-slave-9mr0 krb5kdc[1974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837661, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:54:21.795912 2749 init.cc:377] Logged in from keytab as kudu/127.18.134.2@KRBTEST.COM (short username kudu)
I20260619 02:54:21.797330 2749 webserver.cc:460] Webserver started at http://127.18.134.2:44289/ using document root <none> and password file <none>
I20260619 02:54:21.797991 2749 fs_manager.cc:359] Metadata directory not provided
I20260619 02:54:21.798121 2749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:54:21.803540 2749 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 02:54:21.806207 2764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:54:21.807688 2749 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 02:54:21.807858 2749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "8451d39ceeb948e0814cbdc529dae4aa"
format_stamp: "Formatted at 2026-06-19 02:53:59 on dist-test-slave-9mr0"
server_key: "274da2bed2cf9d78eb59163bf4114c6b"
server_key_iv: "9b89f349b3427a91c1f59b0d3854a4ef"
server_key_version: "encryptionkey@0"
I20260619 02:54:21.808857 2749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:54:21.838867 2749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:54:21.842948 2749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:54:21.843195 2749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:54:21.844137 2749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:54:21.845434 2749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:54:21.845533 2749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:54:21.845633 2749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:54:21.845709 2749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:54:21.876413 2749 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:37295
I20260619 02:54:21.876477 2876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:37295 every 8 connection(s)
I20260619 02:54:21.877872 2749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:54:21.882220 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 2749
I20260619 02:54:21.882850 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:43255
--local_ip_for_outbound_sockets=127.18.134.3
--tserver_master_addrs=127.18.134.62:35511
--webserver_port=39907
--webserver_interface=127.18.134.3
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:45187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 02:54:21 dist-test-slave-9mr0 krb5kdc[1974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837661, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:54:21.897526 2747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:21.880365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:48053 (local address 127.18.134.62:35511)
0619 02:54:21.880543 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 02:54:21.880547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:54:21.881709 (+ 1162us) server_negotiation.cc:407] Connection header received
0619 02:54:21.883673 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:54:21.883679 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:54:21.883773 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:54:21.883859 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:54:21.885419 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:21.886114 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:21.887746 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:21.888172 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:21.888295 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:54:21.892136 (+ 3841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:54:21.892142 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:54:21.892145 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:54:21.892181 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:54:21.894496 (+ 2315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:54:21.895336 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:54:21.895340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:54:21.895342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:54:21.895458 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:54:21.895891 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:54:21.895895 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:54:21.895897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:54:21.896178 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:54:21.896287 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 02:54:21.897087 (+ 800us) server_negotiation.cc:299] Negotiation successful
0619 02:54:21.897269 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:54:21.899019 2879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:21.880848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.2:48053)
0619 02:54:21.881537 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:21.881580 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 02:54:21.883396 (+ 1816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:54:21.884115 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:54:21.884123 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:54:21.884658 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:54:21.885233 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:54:21.885260 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:54:21.886283 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:54:21.886287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:54:21.887584 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:54:21.887608 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:54:21.888417 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:54:21.888425 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:54:21.888644 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:54:21.889279 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:54:21.889301 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:54:21.891877 (+ 2576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:54:21.894625 (+ 2748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:54:21.894664 (+ 39us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:54:21.894682 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:54:21.895223 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:54:21.895542 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:54:21.895546 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:54:21.895549 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:54:21.895777 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:54:21.896292 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:54:21.896299 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:54:21.896724 (+ 425us) client_negotiation.cc:769] Sending connection context
0619 02:54:21.897026 (+ 302us) client_negotiation.cc:240] Negotiation successful
0619 02:54:21.897596 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":265,"threads_started":1}
I20260619 02:54:21.900471 2877 heartbeater.cc:347] Connected to a master server at 127.18.134.62:35511
I20260619 02:54:21.900807 2877 heartbeater.cc:464] Registering TS with master...
I20260619 02:54:21.901526 2877 heartbeater.cc:511] Master 127.18.134.62:35511 requested a full tablet report, sending...
I20260619 02:54:21.902915 2559 ts_manager.cc:194] Registered new tserver with Master: 8451d39ceeb948e0814cbdc529dae4aa (127.18.134.2:37295)
I20260619 02:54:21.903725 2559 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.2@KRBTEST.COM'} at 127.18.134.2:48053
W20260619 02:54:22.205686 2880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:54:22.205982 2880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:54:22.206128 2880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:54:22.213532 2880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:54:22.213666 2880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:54:22.213828 2880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:54:22.223766 2880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:45187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:43255
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=39907
--tserver_master_addrs=127.18.134.62:35511
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:54:22.225013 2880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:54:22.226290 2880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:54:22.238837 2888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:54:22.238863 2886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:54:22.239099 2885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:54:22.239451 2880 server_base.cc:1034] running on GCE node
I20260619 02:54:22.240012 2880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:54:22.240923 2880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:54:22.242129 2880 hybrid_clock.cc:648] HybridClock initialized: now 1781837662242076 us; error 47 us; skew 500 ppm
I20260619 02:54:22.242424 2880 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:54:22 dist-test-slave-9mr0 krb5kdc[1974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837662, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:54:22.245422 2880 init.cc:377] Logged in from keytab as kudu/127.18.134.3@KRBTEST.COM (short username kudu)
I20260619 02:54:22.246789 2880 webserver.cc:460] Webserver started at http://127.18.134.3:39907/ using document root <none> and password file <none>
I20260619 02:54:22.247394 2880 fs_manager.cc:359] Metadata directory not provided
I20260619 02:54:22.247498 2880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:54:22.252466 2880 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260619 02:54:22.255103 2895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:54:22.256335 2880 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260619 02:54:22.256462 2880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "cf9095bb0bd54375a245ae790a8ff346"
format_stamp: "Formatted at 2026-06-19 02:53:59 on dist-test-slave-9mr0"
server_key: "2f5a180c644def0f23a35e6e36963ac5"
server_key_iv: "dea10d9cbb32799a92d5653ca4024041"
server_key_version: "encryptionkey@0"
I20260619 02:54:22.257387 2880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:54:22.278607 2880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:54:22.283102 2880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:54:22.283334 2880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:54:22.284423 2880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:54:22.285804 2880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:54:22.285885 2880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:54:22.285972 2880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:54:22.286026 2880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:54:22.315825 2880 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:43255
I20260619 02:54:22.315879 3007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:43255 every 8 connection(s)
I20260619 02:54:22.317181 2880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:54:22.326076 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 2880
Jun 19 02:54:22 dist-test-slave-9mr0 krb5kdc[1974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837662, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:54:22.338793 2747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:22.319707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:52253 (local address 127.18.134.62:35511)
0619 02:54:22.319926 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 02:54:22.319931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:54:22.320981 (+ 1050us) server_negotiation.cc:407] Connection header received
0619 02:54:22.322780 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:54:22.322786 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:54:22.322868 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:54:22.322956 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:54:22.324430 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.325181 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.326837 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.327379 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.327512 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:54:22.331864 (+ 4352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:54:22.331871 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:54:22.331876 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:54:22.331916 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:54:22.335331 (+ 3415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:54:22.336137 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:54:22.336142 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:54:22.336146 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:54:22.336287 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:54:22.336756 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:54:22.336794 (+ 38us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:54:22.336797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:54:22.337169 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:54:22.337323 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 02:54:22.338108 (+ 785us) server_negotiation.cc:299] Negotiation successful
0619 02:54:22.338382 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260619 02:54:22.339548 3010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:22.320131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.3:52253)
0619 02:54:22.320814 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:22.320849 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:54:22.322492 (+ 1643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:54:22.323197 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:54:22.323205 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:54:22.323722 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:54:22.324259 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:54:22.324277 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.325351 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.325355 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:54:22.326693 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:54:22.326710 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.327857 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.327866 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:54:22.328169 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:54:22.328980 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:54:22.329018 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:54:22.331650 (+ 2632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:54:22.335462 (+ 3812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:54:22.335469 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:54:22.335482 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:54:22.336003 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:54:22.336397 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:54:22.336406 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:54:22.336408 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:54:22.336619 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:54:22.337315 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:54:22.337321 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:54:22.337647 (+ 326us) client_negotiation.cc:769] Sending connection context
0619 02:54:22.337964 (+ 317us) client_negotiation.cc:240] Negotiation successful
0619 02:54:22.338367 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":254,"threads_started":1}
I20260619 02:54:22.341020 3008 heartbeater.cc:347] Connected to a master server at 127.18.134.62:35511
I20260619 02:54:22.341365 3008 heartbeater.cc:464] Registering TS with master...
I20260619 02:54:22.342005 3012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:22.327076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47708 (local address 127.18.134.62:35511)
0619 02:54:22.327913 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0619 02:54:22.327918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:54:22.327936 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:54:22.327983 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:54:22.327988 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:54:22.328056 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:54:22.328162 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:54:22.330214 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.331124 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.332792 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.333305 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.333406 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:54:22.336322 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:54:22.336328 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:54:22.336332 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:54:22.336365 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:54:22.339336 (+ 2971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:54:22.340148 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:54:22.340152 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:54:22.340154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:54:22.340273 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:54:22.340650 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:54:22.340653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:54:22.340655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:54:22.340906 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:54:22.341190 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0619 02:54:22.341425 (+ 235us) server_negotiation.cc:299] Negotiation successful
0619 02:54:22.341641 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":560,"threads_started":1}
I20260619 02:54:22.342262 3008 heartbeater.cc:511] Master 127.18.134.62:35511 requested a full tablet report, sending...
I20260619 02:54:22.343827 2558 ts_manager.cc:194] Registered new tserver with Master: cf9095bb0bd54375a245ae790a8ff346 (127.18.134.3:43255)
I20260619 02:54:22.344801 2558 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.3@KRBTEST.COM'} at 127.18.134.3:52253
I20260619 02:54:22.357133 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:54:22.363933 3012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:22.358060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47712 (local address 127.18.134.62:35511)
0619 02:54:22.358258 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 02:54:22.358263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:54:22.358699 (+ 436us) server_negotiation.cc:407] Connection header received
0619 02:54:22.358899 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:54:22.358902 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:54:22.359063 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:54:22.359204 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:54:22.359654 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.360376 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.362092 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.362535 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.362679 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:54:22.363187 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:54:22.363391 (+ 204us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0619 02:54:22.363703 (+ 312us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47712: token expired
Metrics: {"server-negotiator.queue_time_us":81}
W20260619 02:54:22.364050 3012 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47712: token expired
W20260619 02:54:22.364604 2480 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.18.134.62:35511)
I20260619 02:54:22.376675 3012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:22.364995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47718 (local address 127.18.134.62:35511)
0619 02:54:22.365131 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 02:54:22.365135 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:54:22.365149 (+ 14us) server_negotiation.cc:407] Connection header received
0619 02:54:22.365270 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:54:22.365273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:54:22.365328 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:54:22.365398 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:54:22.366731 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.367454 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.368715 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.369111 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.369205 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:54:22.371846 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:54:22.371850 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:54:22.371854 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:54:22.371886 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:54:22.374430 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:54:22.375112 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:54:22.375116 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:54:22.375118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:54:22.375246 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:54:22.375535 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:54:22.375538 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:54:22.375540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:54:22.375871 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:54:22.376046 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0619 02:54:22.376275 (+ 229us) server_negotiation.cc:299] Negotiation successful
0619 02:54:22.376434 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 02:54:22.380347 2558 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:54:22.387598 2558 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:54:22.411028 3019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:22.398967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36468 (local address 127.18.134.1:41291)
0619 02:54:22.399569 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0619 02:54:22.399578 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:54:22.399847 (+ 269us) server_negotiation.cc:407] Connection header received
0619 02:54:22.400253 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:54:22.400263 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:54:22.400642 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:54:22.400869 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:54:22.401976 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.404172 (+ 2196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.407760 (+ 3588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.409560 (+ 1800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.409743 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:54:22.409994 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0619 02:54:22.410437 (+ 443us) server_negotiation.cc:299] Negotiation successful
0619 02:54:22.410685 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":498,"threads_started":1}
I20260619 02:54:22.411908 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:22.398869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:41291 (local address 127.0.0.1:36468)
0619 02:54:22.399687 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:22.399738 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 02:54:22.399922 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:54:22.401196 (+ 1274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:54:22.401205 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:54:22.401241 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:54:22.401726 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:54:22.401740 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.404386 (+ 2646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.404393 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:54:22.407520 (+ 3127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:54:22.407540 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.409843 (+ 2303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.409850 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:54:22.410116 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:54:22.410177 (+ 61us) client_negotiation.cc:769] Sending connection context
0619 02:54:22.411254 (+ 1077us) client_negotiation.cc:240] Negotiation successful
0619 02:54:22.411508 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":375,"threads_started":1}
I20260619 02:54:22.416127 2680 tablet_service.cc:1467] Processing CreateTablet for tablet 3fdade45edb54040bea2766f8a319459 (DEFAULT_TABLE table=test-table [id=9bf7ac931b9e4bf7b4aaed99628ab01f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:54:22.417752 2680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fdade45edb54040bea2766f8a319459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:54:22.418685 3020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:22.400075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:37295 (local address 127.0.0.1:39670)
0619 02:54:22.400771 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:22.400792 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:54:22.400890 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:54:22.411780 (+ 10890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:54:22.411786 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:54:22.411816 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:54:22.412258 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:54:22.412273 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.414164 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.414170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:54:22.416399 (+ 2229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:54:22.416412 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.417895 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.417902 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:54:22.418094 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:54:22.418118 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 02:54:22.418198 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 02:54:22.418331 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":475,"threads_started":1}
I20260619 02:54:22.418740 3021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:22.400409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39670 (local address 127.18.134.2:37295)
0619 02:54:22.410749 (+ 10340us) server_negotiation.cc:206] Beginning negotiation
0619 02:54:22.410759 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:54:22.410795 (+ 36us) server_negotiation.cc:407] Connection header received
0619 02:54:22.410902 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:54:22.410912 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:54:22.411563 (+ 651us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:54:22.411781 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:54:22.412460 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.414017 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.416641 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.417656 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.417782 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:54:22.417974 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0619 02:54:22.418258 (+ 284us) server_negotiation.cc:299] Negotiation successful
0619 02:54:22.418416 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10132,"thread_start_us":9794,"threads_started":1}
I20260619 02:54:22.420585 3022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:22.400867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:43255 (local address 127.0.0.1:49614)
0619 02:54:22.408786 (+ 7919us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:22.408803 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:54:22.408895 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:54:22.413160 (+ 4265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:54:22.413165 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:54:22.413190 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:54:22.413527 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:54:22.413540 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.415343 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.415349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:54:22.417910 (+ 2561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:54:22.417928 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.419833 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.419839 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:54:22.420010 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:54:22.420044 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 02:54:22.420137 (+ 93us) client_negotiation.cc:240] Negotiation successful
0619 02:54:22.420262 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7789,"thread_start_us":7672,"threads_started":1}
I20260619 02:54:22.421371 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:22.401441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49614 (local address 127.18.134.3:43255)
0619 02:54:22.412689 (+ 11248us) server_negotiation.cc:206] Beginning negotiation
0619 02:54:22.412695 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:54:22.412723 (+ 28us) server_negotiation.cc:407] Connection header received
0619 02:54:22.412796 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:54:22.412802 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:54:22.412980 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:54:22.413158 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:54:22.413718 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.415196 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.418124 (+ 2928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.419602 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.419759 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:54:22.419990 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0619 02:54:22.420857 (+ 867us) server_negotiation.cc:299] Negotiation successful
0619 02:54:22.421016 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11054,"thread_start_us":10902,"threads_started":1}
I20260619 02:54:22.422581 2812 tablet_service.cc:1467] Processing CreateTablet for tablet 3fdade45edb54040bea2766f8a319459 (DEFAULT_TABLE table=test-table [id=9bf7ac931b9e4bf7b4aaed99628ab01f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:54:22.423949 2812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fdade45edb54040bea2766f8a319459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:54:22.430495 2943 tablet_service.cc:1467] Processing CreateTablet for tablet 3fdade45edb54040bea2766f8a319459 (DEFAULT_TABLE table=test-table [id=9bf7ac931b9e4bf7b4aaed99628ab01f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:54:22.431789 2943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fdade45edb54040bea2766f8a319459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:54:22.438212 3024 tablet_bootstrap.cc:492] T 3fdade45edb54040bea2766f8a319459 P 8451d39ceeb948e0814cbdc529dae4aa: Bootstrap starting.
I20260619 02:54:22.441821 3024 tablet_bootstrap.cc:654] T 3fdade45edb54040bea2766f8a319459 P 8451d39ceeb948e0814cbdc529dae4aa: Neither blocks nor log segments found. Creating new log.
I20260619 02:54:22.443038 3024 log.cc:824] T 3fdade45edb54040bea2766f8a319459 P 8451d39ceeb948e0814cbdc529dae4aa: Log is configured to *not* fsync() on all Append() calls
I20260619 02:54:22.443140 3025 tablet_bootstrap.cc:492] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07: Bootstrap starting.
I20260619 02:54:22.446574 3025 tablet_bootstrap.cc:654] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07: Neither blocks nor log segments found. Creating new log.
I20260619 02:54:22.446728 3024 tablet_bootstrap.cc:492] T 3fdade45edb54040bea2766f8a319459 P 8451d39ceeb948e0814cbdc529dae4aa: No bootstrap required, opened a new log
I20260619 02:54:22.446734 3027 tablet_bootstrap.cc:492] T 3fdade45edb54040bea2766f8a319459 P cf9095bb0bd54375a245ae790a8ff346: Bootstrap starting.
I20260619 02:54:22.447042 3024 ts_tablet_manager.cc:1397] T 3fdade45edb54040bea2766f8a319459 P 8451d39ceeb948e0814cbdc529dae4aa: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260619 02:54:22.448078 3025 log.cc:824] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07: Log is configured to *not* fsync() on all Append() calls
I20260619 02:54:22.450030 3027 tablet_bootstrap.cc:654] T 3fdade45edb54040bea2766f8a319459 P cf9095bb0bd54375a245ae790a8ff346: Neither blocks nor log segments found. Creating new log.
I20260619 02:54:22.451365 3027 log.cc:824] T 3fdade45edb54040bea2766f8a319459 P cf9095bb0bd54375a245ae790a8ff346: Log is configured to *not* fsync() on all Append() calls
I20260619 02:54:22.451789 3025 tablet_bootstrap.cc:492] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07: No bootstrap required, opened a new log
I20260619 02:54:22.452101 3025 ts_tablet_manager.cc:1397] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260619 02:54:22.454455 3027 tablet_bootstrap.cc:492] T 3fdade45edb54040bea2766f8a319459 P cf9095bb0bd54375a245ae790a8ff346: No bootstrap required, opened a new log
I20260619 02:54:22.454850 3027 ts_tablet_manager.cc:1397] T 3fdade45edb54040bea2766f8a319459 P cf9095bb0bd54375a245ae790a8ff346: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260619 02:54:22.455590 3024 raft_consensus.cc:348] T 3fdade45edb54040bea2766f8a319459 P 8451d39ceeb948e0814cbdc529dae4aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 41291 } } peers { permanent_uuid: "8451d39ceeb948e0814cbdc529dae4aa" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37295 } } peers { permanent_uuid: "cf9095bb0bd54375a245ae790a8ff346" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43255 } }
I20260619 02:54:22.456043 3024 raft_consensus.cc:374] T 3fdade45edb54040bea2766f8a319459 P 8451d39ceeb948e0814cbdc529dae4aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:54:22.456153 3024 raft_consensus.cc:729] T 3fdade45edb54040bea2766f8a319459 P 8451d39ceeb948e0814cbdc529dae4aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8451d39ceeb948e0814cbdc529dae4aa, State: Initialized, Role: FOLLOWER
I20260619 02:54:22.456776 3024 consensus_queue.cc:260] T 3fdade45edb54040bea2766f8a319459 P 8451d39ceeb948e0814cbdc529dae4aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 41291 } } peers { permanent_uuid: "8451d39ceeb948e0814cbdc529dae4aa" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37295 } } peers { permanent_uuid: "cf9095bb0bd54375a245ae790a8ff346" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43255 } }
I20260619 02:54:22.458797 2877 heartbeater.cc:503] Master 127.18.134.62:35511 was elected leader, sending a full tablet report...
I20260619 02:54:22.459446 3024 ts_tablet_manager.cc:1428] T 3fdade45edb54040bea2766f8a319459 P 8451d39ceeb948e0814cbdc529dae4aa: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260619 02:54:22.462759 3025 raft_consensus.cc:348] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 41291 } } peers { permanent_uuid: "8451d39ceeb948e0814cbdc529dae4aa" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37295 } } peers { permanent_uuid: "cf9095bb0bd54375a245ae790a8ff346" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43255 } }
I20260619 02:54:22.463205 2745 heartbeater.cc:503] Master 127.18.134.62:35511 was elected leader, sending a full tablet report...
I20260619 02:54:22.463279 3025 raft_consensus.cc:374] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:54:22.463408 3025 raft_consensus.cc:729] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2da3d0dd4eb1432787df0d9838ba8b07, State: Initialized, Role: FOLLOWER
I20260619 02:54:22.464071 3025 consensus_queue.cc:260] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 41291 } } peers { permanent_uuid: "8451d39ceeb948e0814cbdc529dae4aa" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37295 } } peers { permanent_uuid: "cf9095bb0bd54375a245ae790a8ff346" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43255 } }
I20260619 02:54:22.466423 3025 ts_tablet_manager.cc:1428] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260619 02:54:22.466583 3027 raft_consensus.cc:348] T 3fdade45edb54040bea2766f8a319459 P cf9095bb0bd54375a245ae790a8ff346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 41291 } } peers { permanent_uuid: "8451d39ceeb948e0814cbdc529dae4aa" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37295 } } peers { permanent_uuid: "cf9095bb0bd54375a245ae790a8ff346" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43255 } }
I20260619 02:54:22.467077 3027 raft_consensus.cc:374] T 3fdade45edb54040bea2766f8a319459 P cf9095bb0bd54375a245ae790a8ff346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:54:22.467195 3027 raft_consensus.cc:729] T 3fdade45edb54040bea2766f8a319459 P cf9095bb0bd54375a245ae790a8ff346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf9095bb0bd54375a245ae790a8ff346, State: Initialized, Role: FOLLOWER
I20260619 02:54:22.467844 3027 consensus_queue.cc:260] T 3fdade45edb54040bea2766f8a319459 P cf9095bb0bd54375a245ae790a8ff346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 41291 } } peers { permanent_uuid: "8451d39ceeb948e0814cbdc529dae4aa" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37295 } } peers { permanent_uuid: "cf9095bb0bd54375a245ae790a8ff346" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43255 } }
I20260619 02:54:22.468924 3008 heartbeater.cc:503] Master 127.18.134.62:35511 was elected leader, sending a full tablet report...
I20260619 02:54:22.469471 3027 ts_tablet_manager.cc:1428] T 3fdade45edb54040bea2766f8a319459 P cf9095bb0bd54375a245ae790a8ff346: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260619 02:54:22.568689 3009 tablet.cc:2369] T 3fdade45edb54040bea2766f8a319459 P cf9095bb0bd54375a245ae790a8ff346: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:54:22.628787 3031 raft_consensus.cc:482] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:54:22.629081 3031 raft_consensus.cc:504] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 41291 } } peers { permanent_uuid: "8451d39ceeb948e0814cbdc529dae4aa" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37295 } } peers { permanent_uuid: "cf9095bb0bd54375a245ae790a8ff346" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43255 } }
W20260619 02:54:22.629774 2878 tablet.cc:2369] T 3fdade45edb54040bea2766f8a319459 P 8451d39ceeb948e0814cbdc529dae4aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:54:22.630995 3031 leader_election.cc:288] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8451d39ceeb948e0814cbdc529dae4aa (127.18.134.2:37295), cf9095bb0bd54375a245ae790a8ff346 (127.18.134.3:43255)
I20260619 02:54:22.637802 3021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:22.631357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:60763 (local address 127.18.134.2:37295)
0619 02:54:22.631526 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 02:54:22.631530 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:54:22.632109 (+ 579us) server_negotiation.cc:407] Connection header received
0619 02:54:22.632333 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:54:22.632339 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:54:22.632405 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:54:22.632508 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:54:22.633149 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.633865 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.635656 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.636770 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.636925 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:54:22.636999 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 02:54:22.637406 (+ 407us) server_negotiation.cc:299] Negotiation successful
0619 02:54:22.637550 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 02:54:22.637884 3033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:22.631344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:37295 (local address 127.18.134.1:60763)
0619 02:54:22.632006 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:22.632024 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:54:22.632167 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:54:22.632552 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:54:22.632557 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:54:22.632593 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:54:22.633006 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:54:22.633017 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.633997 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.634001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:54:22.635496 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:54:22.635509 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.636993 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.636999 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:54:22.637170 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:54:22.637201 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 02:54:22.637353 (+ 152us) client_negotiation.cc:240] Negotiation successful
0619 02:54:22.637476 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":334,"threads_started":1}
I20260619 02:54:22.638466 3034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:22.632140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:43255 (local address 127.18.134.1:47227)
0619 02:54:22.632970 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:22.632983 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:54:22.633095 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:54:22.633363 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:54:22.633367 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:54:22.633388 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:54:22.633668 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:54:22.633676 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.634734 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.634740 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:54:22.636203 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:54:22.636219 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.637801 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.637808 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:54:22.637980 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:54:22.638013 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 02:54:22.638101 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 02:54:22.638223 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"mutex_wait_us":4,"thread_start_us":511,"threads_started":1}
I20260619 02:54:22.638774 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:22.632264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:47227 (local address 127.18.134.3:43255)
0619 02:54:22.632484 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0619 02:54:22.632491 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:54:22.633065 (+ 574us) server_negotiation.cc:407] Connection header received
0619 02:54:22.633201 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:54:22.633206 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:54:22.633267 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:54:22.633338 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:54:22.633809 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.634600 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.636398 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.637594 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.637748 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:54:22.637828 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 02:54:22.638394 (+ 566us) server_negotiation.cc:299] Negotiation successful
0619 02:54:22.638521 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 02:54:22.638885 2832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fdade45edb54040bea2766f8a319459" candidate_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8451d39ceeb948e0814cbdc529dae4aa" is_pre_election: true
I20260619 02:54:22.639439 2832 raft_consensus.cc:2454] T 3fdade45edb54040bea2766f8a319459 P 8451d39ceeb948e0814cbdc529dae4aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2da3d0dd4eb1432787df0d9838ba8b07 in term 0.
I20260619 02:54:22.639644 2963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fdade45edb54040bea2766f8a319459" candidate_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf9095bb0bd54375a245ae790a8ff346" is_pre_election: true
I20260619 02:54:22.640120 2963 raft_consensus.cc:2454] T 3fdade45edb54040bea2766f8a319459 P cf9095bb0bd54375a245ae790a8ff346 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2da3d0dd4eb1432787df0d9838ba8b07 in term 0.
I20260619 02:54:22.640153 2633 leader_election.cc:302] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2da3d0dd4eb1432787df0d9838ba8b07, 8451d39ceeb948e0814cbdc529dae4aa; no voters:
I20260619 02:54:22.640533 3031 raft_consensus.cc:2781] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:54:22.640664 3031 raft_consensus.cc:482] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:54:22.640762 3031 raft_consensus.cc:3037] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:54:22.642946 3031 raft_consensus.cc:504] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 41291 } } peers { permanent_uuid: "8451d39ceeb948e0814cbdc529dae4aa" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37295 } } peers { permanent_uuid: "cf9095bb0bd54375a245ae790a8ff346" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43255 } }
I20260619 02:54:22.643502 3031 leader_election.cc:288] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07 [CANDIDATE]: Term 1 election: Requested vote from peers 8451d39ceeb948e0814cbdc529dae4aa (127.18.134.2:37295), cf9095bb0bd54375a245ae790a8ff346 (127.18.134.3:43255)
I20260619 02:54:22.644016 2832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fdade45edb54040bea2766f8a319459" candidate_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8451d39ceeb948e0814cbdc529dae4aa"
I20260619 02:54:22.644016 2963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fdade45edb54040bea2766f8a319459" candidate_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf9095bb0bd54375a245ae790a8ff346"
I20260619 02:54:22.644218 2963 raft_consensus.cc:3037] T 3fdade45edb54040bea2766f8a319459 P cf9095bb0bd54375a245ae790a8ff346 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:54:22.644218 2832 raft_consensus.cc:3037] T 3fdade45edb54040bea2766f8a319459 P 8451d39ceeb948e0814cbdc529dae4aa [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:54:22.647308 2963 raft_consensus.cc:2454] T 3fdade45edb54040bea2766f8a319459 P cf9095bb0bd54375a245ae790a8ff346 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2da3d0dd4eb1432787df0d9838ba8b07 in term 1.
I20260619 02:54:22.647313 2832 raft_consensus.cc:2454] T 3fdade45edb54040bea2766f8a319459 P 8451d39ceeb948e0814cbdc529dae4aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2da3d0dd4eb1432787df0d9838ba8b07 in term 1.
I20260619 02:54:22.647733 2633 leader_election.cc:302] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2da3d0dd4eb1432787df0d9838ba8b07, 8451d39ceeb948e0814cbdc529dae4aa; no voters:
I20260619 02:54:22.647998 3031 raft_consensus.cc:2781] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:54:22.648411 3031 raft_consensus.cc:686] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07 [term 1 LEADER]: Becoming Leader. State: Replica: 2da3d0dd4eb1432787df0d9838ba8b07, State: Running, Role: LEADER
I20260619 02:54:22.648855 3031 consensus_queue.cc:237] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 41291 } } peers { permanent_uuid: "8451d39ceeb948e0814cbdc529dae4aa" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37295 } } peers { permanent_uuid: "cf9095bb0bd54375a245ae790a8ff346" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43255 } }
I20260619 02:54:22.652721 2558 catalog_manager.cc:5495] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2da3d0dd4eb1432787df0d9838ba8b07 (127.18.134.1). New cstate: current_term: 1 leader_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2da3d0dd4eb1432787df0d9838ba8b07" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 41291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8451d39ceeb948e0814cbdc529dae4aa" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf9095bb0bd54375a245ae790a8ff346" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43255 } health_report { overall_health: UNKNOWN } } }
W20260619 02:54:22.677948 2746 tablet.cc:2369] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:54:22.706478 3019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:22.701030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36470 (local address 127.18.134.1:41291)
0619 02:54:22.701255 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0619 02:54:22.701260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:54:22.701278 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:54:22.701367 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:54:22.701371 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:54:22.701449 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:54:22.701576 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:54:22.701944 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.702777 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.704515 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.704931 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.705024 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:54:22.705475 (+ 451us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:54:22.705994 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:54:22.706098 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 02:54:22.706200 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 02:54:22.706267 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 02:54:22.714788 2963 raft_consensus.cc:1261] T 3fdade45edb54040bea2766f8a319459 P cf9095bb0bd54375a245ae790a8ff346 [term 1 FOLLOWER]: Refusing update from remote peer 2da3d0dd4eb1432787df0d9838ba8b07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:54:22.714792 2832 raft_consensus.cc:1261] T 3fdade45edb54040bea2766f8a319459 P 8451d39ceeb948e0814cbdc529dae4aa [term 1 FOLLOWER]: Refusing update from remote peer 2da3d0dd4eb1432787df0d9838ba8b07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:54:22.715698 3031 consensus_queue.cc:1035] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf9095bb0bd54375a245ae790a8ff346" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:54:22.715996 3035 consensus_queue.cc:1035] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8451d39ceeb948e0814cbdc529dae4aa" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 37295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:54:22.725553 3037 mvcc.cc:205] Tried to move back new op lower bound from 7298407066469707776 to 7298407066213089280. Current Snapshot: MvccSnapshot[applied={T|T < 7298407066469707776}]
I20260619 02:54:22.741317 2557 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:54:22.741667 2557 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:54:22.745498 2557 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3: Sending DeleteTablet for 3 replicas of tablet 3fdade45edb54040bea2766f8a319459
I20260619 02:54:22.746750 2680 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fdade45edb54040bea2766f8a319459 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:54:22 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:36468
I20260619 02:54:22.746953 2812 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fdade45edb54040bea2766f8a319459 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:54:22 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:39670
I20260619 02:54:22.747432 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 2596
I20260619 02:54:22.747673 2943 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fdade45edb54040bea2766f8a319459 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:54:22 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:49614
I20260619 02:54:22.748538 3048 tablet_replica.cc:330] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07: stopping tablet replica
I20260619 02:54:22.748874 3049 tablet_replica.cc:330] T 3fdade45edb54040bea2766f8a319459 P 8451d39ceeb948e0814cbdc529dae4aa: stopping tablet replica
I20260619 02:54:22.749094 3048 raft_consensus.cc:2229] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:54:22.749265 3049 raft_consensus.cc:2229] T 3fdade45edb54040bea2766f8a319459 P 8451d39ceeb948e0814cbdc529dae4aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:54:22.749768 3049 raft_consensus.cc:2258] T 3fdade45edb54040bea2766f8a319459 P 8451d39ceeb948e0814cbdc529dae4aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:54:22.749903 3048 raft_consensus.cc:2258] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:54:22.752007 3049 ts_tablet_manager.cc:1905] T 3fdade45edb54040bea2766f8a319459 P 8451d39ceeb948e0814cbdc529dae4aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:54:22.757768 3048 ts_tablet_manager.cc:1905] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:54:22.759343 3050 tablet_replica.cc:330] T 3fdade45edb54040bea2766f8a319459 P cf9095bb0bd54375a245ae790a8ff346: stopping tablet replica
I20260619 02:54:22.759783 3050 raft_consensus.cc:2229] T 3fdade45edb54040bea2766f8a319459 P cf9095bb0bd54375a245ae790a8ff346 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:54:22.760346 3050 raft_consensus.cc:2258] T 3fdade45edb54040bea2766f8a319459 P cf9095bb0bd54375a245ae790a8ff346 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:54:22.762326 3050 ts_tablet_manager.cc:1905] T 3fdade45edb54040bea2766f8a319459 P cf9095bb0bd54375a245ae790a8ff346: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:54:22.766598 3048 ts_tablet_manager.cc:1918] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:54:22.766837 3048 log.cc:1196] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/wal/wals/3fdade45edb54040bea2766f8a319459
I20260619 02:54:22.767408 3048 ts_tablet_manager.cc:1939] T 3fdade45edb54040bea2766f8a319459 P 2da3d0dd4eb1432787df0d9838ba8b07: Deleting consensus metadata
I20260619 02:54:22.767444 3019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:22.748661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36480 (local address 127.18.134.1:41291)
0619 02:54:22.748812 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 02:54:22.748817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:54:22.748833 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:54:22.748883 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:54:22.748889 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:54:22.748958 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:54:22.749054 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:54:22.750955 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.752232 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.754353 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.755099 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.755223 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:54:22.758833 (+ 3610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:54:22.758848 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:54:22.758867 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:54:22.758921 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:54:22.763185 (+ 4264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:54:22.764137 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:54:22.764146 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:54:22.764153 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:54:22.764330 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:54:22.764782 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:54:22.764791 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:54:22.764794 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:54:22.765250 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:54:22.765592 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0619 02:54:22.766832 (+ 1240us) server_negotiation.cc:299] Negotiation successful
0619 02:54:22.767072 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 02:54:22.767624 3049 ts_tablet_manager.cc:1918] T 3fdade45edb54040bea2766f8a319459 P 8451d39ceeb948e0814cbdc529dae4aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:54:22.767823 3049 log.cc:1196] T 3fdade45edb54040bea2766f8a319459 P 8451d39ceeb948e0814cbdc529dae4aa: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/wal/wals/3fdade45edb54040bea2766f8a319459
I20260619 02:54:22.768357 3049 ts_tablet_manager.cc:1939] T 3fdade45edb54040bea2766f8a319459 P 8451d39ceeb948e0814cbdc529dae4aa: Deleting consensus metadata
I20260619 02:54:22.768474 2740 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:54:22.770082 2543 catalog_manager.cc:4841] TS 8451d39ceeb948e0814cbdc529dae4aa (127.18.134.2:37295): tablet 3fdade45edb54040bea2766f8a319459 (table test-table [id=9bf7ac931b9e4bf7b4aaed99628ab01f]) successfully deleted
I20260619 02:54:22.770565 3050 ts_tablet_manager.cc:1918] T 3fdade45edb54040bea2766f8a319459 P cf9095bb0bd54375a245ae790a8ff346: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:54:22.770761 3050 log.cc:1196] T 3fdade45edb54040bea2766f8a319459 P cf9095bb0bd54375a245ae790a8ff346: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/wal/wals/3fdade45edb54040bea2766f8a319459
I20260619 02:54:22.771247 3050 ts_tablet_manager.cc:1939] T 3fdade45edb54040bea2766f8a319459 P cf9095bb0bd54375a245ae790a8ff346: Deleting consensus metadata
I20260619 02:54:22.772897 2543 catalog_manager.cc:4841] TS cf9095bb0bd54375a245ae790a8ff346 (127.18.134.3:43255): tablet 3fdade45edb54040bea2766f8a319459 (table test-table [id=9bf7ac931b9e4bf7b4aaed99628ab01f]) successfully deleted
I20260619 02:54:22.865847 2543 catalog_manager.cc:4841] TS 2da3d0dd4eb1432787df0d9838ba8b07 (127.18.134.1:41291): tablet 3fdade45edb54040bea2766f8a319459 (table test-table [id=9bf7ac931b9e4bf7b4aaed99628ab01f]) successfully deleted
I20260619 02:54:22.867034 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 2596
I20260619 02:54:22.881788 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 2749
I20260619 02:54:22.899169 3021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:22.882558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39680 (local address 127.18.134.2:37295)
0619 02:54:22.882760 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 02:54:22.882766 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:54:22.882787 (+ 21us) server_negotiation.cc:407] Connection header received
0619 02:54:22.882883 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:54:22.882889 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:54:22.882988 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:54:22.883084 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:54:22.884323 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.885464 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.886849 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:22.887476 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:22.887596 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:54:22.893957 (+ 6361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:54:22.893969 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:54:22.893982 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:54:22.894020 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:54:22.896775 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:54:22.897444 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:54:22.897450 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:54:22.897452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:54:22.897570 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:54:22.897884 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:54:22.897887 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:54:22.897889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:54:22.898272 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:54:22.898475 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0619 02:54:22.898719 (+ 244us) server_negotiation.cc:299] Negotiation successful
0619 02:54:22.898881 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 02:54:22.899945 2872 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:54:22.988336 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 2749
I20260619 02:54:23.001711 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 2880
I20260619 02:54:23.015444 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:23.002436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49620 (local address 127.18.134.3:43255)
0619 02:54:23.002646 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 02:54:23.002653 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:54:23.002676 (+ 23us) server_negotiation.cc:407] Connection header received
0619 02:54:23.002777 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:54:23.002781 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:54:23.002849 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:54:23.002961 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:54:23.004697 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:23.005436 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:23.007036 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:54:23.007442 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:54:23.007547 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:54:23.010138 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:54:23.010155 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:54:23.010172 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:54:23.010209 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:54:23.012886 (+ 2677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:54:23.013585 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:54:23.013592 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:54:23.013595 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:54:23.013758 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:54:23.014086 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:54:23.014088 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:54:23.014090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:54:23.014410 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:54:23.014685 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0619 02:54:23.015001 (+ 316us) server_negotiation.cc:299] Negotiation successful
0619 02:54:23.015175 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 02:54:23.016110 3003 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:54:23.105257 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 2880
I20260619 02:54:23.118225 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 2526
I20260619 02:54:23.119429 2589 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:54:23.232389 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 2526
I20260619 02:54:43.250617 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:35511
--webserver_interface=127.18.134.62
--webserver_port=45413
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:45187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:35511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:54:43.569312 3055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:54:43.569603 3055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:54:43.569705 3055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:54:43.576912 3055 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 02:54:43.577041 3055 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 02:54:43.577081 3055 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 02:54:43.577133 3055 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:54:43.577190 3055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:54:43.577257 3055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:54:43.577301 3055 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:54:43.577359 3055 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:54:43.587262 3055 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:45187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:35511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:35511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=45413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:54:43.588497 3055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:54:43.589798 3055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:54:43.601501 3060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:54:43.601449 3061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:54:43.601536 3063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:54:43.602129 3055 server_base.cc:1034] running on GCE node
I20260619 02:54:43.602924 3055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:54:43.604265 3055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:54:43.605501 3055 hybrid_clock.cc:648] HybridClock initialized: now 1781837683605428 us; error 38 us; skew 500 ppm
I20260619 02:54:43.605822 3055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:54:43 dist-test-slave-9mr0 krb5kdc[1974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837683, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:54:43.613734 3055 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:54:43.615288 3055 webserver.cc:460] Webserver started at http://127.18.134.62:45413/ using document root <none> and password file <none>
I20260619 02:54:43.615999 3055 fs_manager.cc:359] Metadata directory not provided
I20260619 02:54:43.616106 3055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:54:43.621601 3055 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260619 02:54:43.624451 3070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:54:43.625744 3055 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 02:54:43.625928 3055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "2d04b4e2748641509554c98f7efecfc3"
format_stamp: "Formatted at 2026-06-19 02:53:58 on dist-test-slave-9mr0"
server_key: "c5584d847587e46c08071fd2eb906d26"
server_key_iv: "43001b69266fabf6ed8d4ca2444ee469"
server_key_version: "encryptionkey@0"
I20260619 02:54:43.626930 3055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:54:43.647396 3055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:54:43.651278 3055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:54:43.651536 3055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:54:43.668470 3055 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:35511
I20260619 02:54:43.668502 3121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:35511 every 8 connection(s)
I20260619 02:54:43.669894 3055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:54:43.672545 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 3055
I20260619 02:54:43.673481 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:41291
--local_ip_for_outbound_sockets=127.18.134.1
--tserver_master_addrs=127.18.134.62:35511
--webserver_port=38429
--webserver_interface=127.18.134.1
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:45187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:54:43.678525 3122 sys_catalog.cc:263] Verifying existing consensus state
I20260619 02:54:43.682919 3122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3: Bootstrap starting.
I20260619 02:54:43.702874 3122 log.cc:824] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3: Log is configured to *not* fsync() on all Append() calls
I20260619 02:54:43.718466 3122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 02:54:43.719044 3122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3: Bootstrap complete.
I20260619 02:54:43.727797 3122 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d04b4e2748641509554c98f7efecfc3" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35511 } }
I20260619 02:54:43.728658 3122 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d04b4e2748641509554c98f7efecfc3, State: Initialized, Role: FOLLOWER
I20260619 02:54:43.729281 3122 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d04b4e2748641509554c98f7efecfc3" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35511 } }
I20260619 02:54:43.729501 3122 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:54:43.729607 3122 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:54:43.729753 3122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [term 2 FOLLOWER]: Advancing to term 3
I20260619 02:54:43.733680 3122 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d04b4e2748641509554c98f7efecfc3" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35511 } }
I20260619 02:54:43.734134 3122 leader_election.cc:302] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d04b4e2748641509554c98f7efecfc3; no voters:
I20260619 02:54:43.734764 3122 leader_election.cc:288] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 02:54:43.734895 3126 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 02:54:43.736102 3126 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [term 3 LEADER]: Becoming Leader. State: Replica: 2d04b4e2748641509554c98f7efecfc3, State: Running, Role: LEADER
I20260619 02:54:43.736708 3126 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d04b4e2748641509554c98f7efecfc3" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35511 } }
I20260619 02:54:43.737145 3122 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:54:43.739360 3128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d04b4e2748641509554c98f7efecfc3. Latest consensus state: current_term: 3 leader_uuid: "2d04b4e2748641509554c98f7efecfc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d04b4e2748641509554c98f7efecfc3" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35511 } } }
I20260619 02:54:43.739394 3127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "2d04b4e2748641509554c98f7efecfc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d04b4e2748641509554c98f7efecfc3" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35511 } } }
I20260619 02:54:43.739754 3128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [sys.catalog]: This master's current role is: LEADER
I20260619 02:54:43.739754 3127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3 [sys.catalog]: This master's current role is: LEADER
I20260619 02:54:43.749130 3133 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:54:43.757812 3133 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:54:43.759353 3133 catalog_manager.cc:1259] Loaded cluster ID: 070be589d2284f839da167d7b368c425
I20260619 02:54:43.759511 3133 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:54:43.763605 3133 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:54:43.765267 3133 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3: Loaded TSK: 0
I20260619 02:54:43.765501 3133 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2d04b4e2748641509554c98f7efecfc3: Loaded TSK: 1
F20260619 02:54:43.766667 3133 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fbe499f59ad google::LogMessage::Fail() at ??:0
@ 0x7fbe499f888a google::LogMessage::SendToLog() at ??:0
@ 0x7fbe499f54de google::LogMessage::Flush() at ??:0
@ 0x7fbe499f6a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fbe4d7597d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fbe4d74873e kudu::security::TokenSigner::ImportKeys() at ??:0
I20260619 02:54:43.785216 3144 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7fbe5f4241c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fbe5f423783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fbe5f4af2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fbe5f4af0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fbe5f4f95e9 std::function<>::operator()() at ??:0
@ 0x7fbe5f427cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fbe5f425c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fbe5f4ad610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fbe5f4ad439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fbe4c641345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fbe4c64491c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fbe4c644799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fbe4c61feaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fbe4f8f96db start_thread at ??:0
@ 0x7fbe4888f61f clone at ??:0
W20260619 02:54:44.050292 3124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:54:44.050621 3124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:54:44.050786 3124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:54:44.058367 3124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:54:44.058488 3124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:54:44.058653 3124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:54:44.069020 3124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:45187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:41291
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=38429
--tserver_master_addrs=127.18.134.62:35511
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:54:44.070341 3124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:54:44.071707 3124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:54:44.085098 3170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:54:44.085086 3169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:54:44.085093 3172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:54:44.085685 3124 server_base.cc:1034] running on GCE node
I20260619 02:54:44.086686 3124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:54:44.087637 3124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:54:44.088846 3124 hybrid_clock.cc:648] HybridClock initialized: now 1781837684088804 us; error 31 us; skew 500 ppm
I20260619 02:54:44.089157 3124 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:54:44 dist-test-slave-9mr0 krb5kdc[1974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837684, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:54:44.092263 3124 init.cc:377] Logged in from keytab as kudu/127.18.134.1@KRBTEST.COM (short username kudu)
I20260619 02:54:44.093658 3124 webserver.cc:460] Webserver started at http://127.18.134.1:38429/ using document root <none> and password file <none>
I20260619 02:54:44.094308 3124 fs_manager.cc:359] Metadata directory not provided
I20260619 02:54:44.094417 3124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:54:44.099810 3124 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260619 02:54:44.104238 3179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:54:44.105685 3124 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 02:54:44.105844 3124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "2da3d0dd4eb1432787df0d9838ba8b07"
format_stamp: "Formatted at 2026-06-19 02:53:58 on dist-test-slave-9mr0"
server_key: "3ba52d3a1c4e3e634d225f1a0b82e3ff"
server_key_iv: "0d4817d0a8d4f845f93f57ad7aaa1564"
server_key_version: "encryptionkey@0"
I20260619 02:54:44.106913 3124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:54:44.122820 3124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:54:44.127113 3124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:54:44.127386 3124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:54:44.128378 3124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:54:44.129705 3124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:54:44.129792 3124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:54:44.129886 3124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:54:44.129963 3124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:54:44.161095 3124 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:41291
I20260619 02:54:44.161147 3291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:41291 every 8 connection(s)
I20260619 02:54:44.162503 3124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:54:44.167204 3294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:44.165313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.1:60281)
0619 02:54:44.166045 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:44.166270 (+ 225us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":298,"threads_started":1}
W20260619 02:54:44.167879 3292 heartbeater.cc:644] Failed to heartbeat to 127.18.134.62:35511 (0 consecutive failures): Network error: Failed to ping master at 127.18.134.62:35511: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
I20260619 02:54:44.168715 3294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:44.168393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.1:45835)
0619 02:54:44.168533 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:44.168607 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 02:54:44.168946 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 3124
I20260619 02:54:44.169492 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:37295
--local_ip_for_outbound_sockets=127.18.134.2
--tserver_master_addrs=127.18.134.62:35511
--webserver_port=44289
--webserver_interface=127.18.134.2
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:45187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:54:44.169569 3294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:44.169249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.1:46185)
0619 02:54:44.169391 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:44.169464 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260619 02:54:44.169853 3292 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:54:44.482259 3295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:54:44.482537 3295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:54:44.482656 3295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:54:44.490900 3295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:54:44.491014 3295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:54:44.491135 3295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:54:44.500905 3295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:45187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:37295
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=44289
--tserver_master_addrs=127.18.134.62:35511
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:54:44.502110 3295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:54:44.503387 3295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:54:44.516088 3300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:54:44.516150 3301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:54:44.516150 3303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:54:44.516616 3295 server_base.cc:1034] running on GCE node
I20260619 02:54:44.517439 3295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:54:44.518357 3295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:54:44.519568 3295 hybrid_clock.cc:648] HybridClock initialized: now 1781837684519487 us; error 66 us; skew 500 ppm
I20260619 02:54:44.519886 3295 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:54:44 dist-test-slave-9mr0 krb5kdc[1974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837684, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:54:44.522989 3295 init.cc:377] Logged in from keytab as kudu/127.18.134.2@KRBTEST.COM (short username kudu)
I20260619 02:54:44.524391 3295 webserver.cc:460] Webserver started at http://127.18.134.2:44289/ using document root <none> and password file <none>
I20260619 02:54:44.525105 3295 fs_manager.cc:359] Metadata directory not provided
I20260619 02:54:44.525224 3295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:54:44.530803 3295 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 02:54:44.534014 3310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:54:44.535487 3295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:54:44.535658 3295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "8451d39ceeb948e0814cbdc529dae4aa"
format_stamp: "Formatted at 2026-06-19 02:53:59 on dist-test-slave-9mr0"
server_key: "274da2bed2cf9d78eb59163bf4114c6b"
server_key_iv: "9b89f349b3427a91c1f59b0d3854a4ef"
server_key_version: "encryptionkey@0"
I20260619 02:54:44.536700 3295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:54:44.548305 3295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:54:44.552366 3295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:54:44.552628 3295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:54:44.553603 3295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:54:44.554948 3295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:54:44.555042 3295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:54:44.555150 3295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:54:44.555217 3295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:54:44.585551 3295 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:37295
I20260619 02:54:44.585615 3422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:37295 every 8 connection(s)
I20260619 02:54:44.587052 3295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:54:44.590701 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 3295
I20260619 02:54:44.591358 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:43255
--local_ip_for_outbound_sockets=127.18.134.3
--tserver_master_addrs=127.18.134.62:35511
--webserver_port=39907
--webserver_interface=127.18.134.3
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:45187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:54:44.591773 3425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:44.589847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.2:52755)
0619 02:54:44.590548 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:44.590835 (+ 287us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":263,"threads_started":1}
W20260619 02:54:44.592440 3423 heartbeater.cc:644] Failed to heartbeat to 127.18.134.62:35511 (0 consecutive failures): Network error: Failed to ping master at 127.18.134.62:35511: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
I20260619 02:54:44.593287 3425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:44.592970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.2:45959)
0619 02:54:44.593110 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:44.593179 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260619 02:54:44.594101 3425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:44.593784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.2:56667)
0619 02:54:44.593918 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:44.593997 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260619 02:54:44.594326 3423 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 02:54:44.906800 3426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:54:44.907078 3426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:54:44.907171 3426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:54:44.914600 3426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:54:44.914748 3426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:54:44.914875 3426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:54:44.924727 3426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:45187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:43255
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=39907
--tserver_master_addrs=127.18.134.62:35511
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:54:44.926029 3426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:54:44.927322 3426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:54:44.939982 3432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:54:44.939960 3431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:54:44.939939 3434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:54:44.940778 3426 server_base.cc:1034] running on GCE node
I20260619 02:54:44.941419 3426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:54:44.942391 3426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:54:44.943629 3426 hybrid_clock.cc:648] HybridClock initialized: now 1781837684943556 us; error 55 us; skew 500 ppm
I20260619 02:54:44.943959 3426 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:54:44 dist-test-slave-9mr0 krb5kdc[1974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837684, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:54:44.947191 3426 init.cc:377] Logged in from keytab as kudu/127.18.134.3@KRBTEST.COM (short username kudu)
I20260619 02:54:44.948642 3426 webserver.cc:460] Webserver started at http://127.18.134.3:39907/ using document root <none> and password file <none>
I20260619 02:54:44.949296 3426 fs_manager.cc:359] Metadata directory not provided
I20260619 02:54:44.949399 3426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:54:44.955065 3426 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 02:54:44.957859 3441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:54:44.959410 3426 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:54:44.959590 3426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "cf9095bb0bd54375a245ae790a8ff346"
format_stamp: "Formatted at 2026-06-19 02:53:59 on dist-test-slave-9mr0"
server_key: "2f5a180c644def0f23a35e6e36963ac5"
server_key_iv: "dea10d9cbb32799a92d5653ca4024041"
server_key_version: "encryptionkey@0"
I20260619 02:54:44.960582 3426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:54:44.972084 3426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:54:44.976277 3426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:54:44.976518 3426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:54:44.977453 3426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:54:44.978820 3426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:54:44.978916 3426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:54:44.978996 3426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:54:44.979048 3426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:54:45.011634 3426 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:43255
I20260619 02:54:45.011684 3553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:43255 every 8 connection(s)
I20260619 02:54:45.013068 3426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:54:45.013665 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 3426
I20260619 02:54:45.020155 3557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:45.017467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.3:49613)
0619 02:54:45.018488 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:45.018822 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":881,"threads_started":1}
W20260619 02:54:45.020916 3554 heartbeater.cc:644] Failed to heartbeat to 127.18.134.62:35511 (0 consecutive failures): Network error: Failed to ping master at 127.18.134.62:35511: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
I20260619 02:54:45.021988 3557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:45.021543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.3:54481)
0619 02:54:45.021727 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:45.021835 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 02:54:45.023244 3557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:45.022836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.3:49291)
0619 02:54:45.023001 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:45.023099 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260619 02:54:45.023541 3554 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 02:54:45.171244 3558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:45.170426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.1:59753)
0619 02:54:45.170903 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:45.171028 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":246,"threads_started":1}
I20260619 02:54:45.595660 3559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:45.594894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.2:57765)
0619 02:54:45.595353 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:45.595434 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":215,"threads_started":1}
I20260619 02:54:46.024897 3560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:46.024109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.3:43905)
0619 02:54:46.024587 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:46.024671 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":235,"threads_started":1}
I20260619 02:54:46.172979 3561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:46.172200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.1:46313)
0619 02:54:46.172703 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:46.172799 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":261,"threads_started":1}
I20260619 02:54:46.597517 3562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:46.596675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.2:44373)
0619 02:54:46.597219 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:46.597315 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":280,"threads_started":1}
I20260619 02:54:47.026885 3563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:47.025965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.3:60879)
0619 02:54:47.026540 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:47.026700 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":299,"threads_started":1}
I20260619 02:54:47.174772 3564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:47.173929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.1:39315)
0619 02:54:47.174450 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:47.174537 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":293,"threads_started":1}
I20260619 02:54:47.599218 3565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:47.598487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.2:53159)
0619 02:54:47.598962 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:47.599047 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":246,"threads_started":1}
I20260619 02:54:48.028599 3566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:48.027858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.3:60873)
0619 02:54:48.028350 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:48.028433 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":272,"threads_started":1}
I20260619 02:54:48.176347 3567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:48.175648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.1:42789)
0619 02:54:48.176073 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:48.176154 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":212,"threads_started":1}
I20260619 02:54:48.600808 3568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:48.600076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.2:56833)
0619 02:54:48.600521 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:48.600600 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":212,"threads_started":1}
I20260619 02:54:49.030275 3569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:49.029453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.3:33171)
0619 02:54:49.029954 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:49.030055 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":257,"threads_started":1}
I20260619 02:54:49.177989 3570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:49.177280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.1:54155)
0619 02:54:49.177738 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:49.177824 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":234,"threads_started":1}
I20260619 02:54:49.602501 3571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:49.601763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.2:48781)
0619 02:54:49.602239 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:49.602326 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":243,"threads_started":1}
I20260619 02:54:50.031913 3572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:50.031204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.3:50769)
0619 02:54:50.031651 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:50.031734 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":216,"threads_started":1}
I20260619 02:54:50.179726 3573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:50.178948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.1:47779)
0619 02:54:50.179458 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:50.179561 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":278,"threads_started":1}
I20260619 02:54:50.604168 3574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:50.603417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.2:36659)
0619 02:54:50.603885 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:50.603994 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":226,"threads_started":1}
I20260619 02:54:51.033489 3575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:51.032756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.3:59015)
0619 02:54:51.033221 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:51.033320 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":245,"threads_started":1}
I20260619 02:54:51.181478 3576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:51.180674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.1:47041)
0619 02:54:51.181165 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:51.181253 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":199,"threads_started":1}
I20260619 02:54:51.605822 3577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:51.605049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.2:39105)
0619 02:54:51.605527 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:51.605610 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":212,"threads_started":1}
I20260619 02:54:52.035526 3578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:52.034547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.3:41153)
0619 02:54:52.035140 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:52.035275 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":300,"threads_started":1}
I20260619 02:54:52.183346 3579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:52.182561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.1:58707)
0619 02:54:52.183074 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:52.183168 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":267,"threads_started":1}
I20260619 02:54:52.607509 3580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:52.606798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.2:41041)
0619 02:54:52.607235 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:52.607320 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":210,"threads_started":1}
I20260619 02:54:53.037448 3581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:53.036644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.3:60039)
0619 02:54:53.037134 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:53.037231 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":253,"threads_started":1}
I20260619 02:54:53.185506 3582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:53.184430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.1:44091)
0619 02:54:53.184995 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:53.185107 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":241,"threads_started":1}
I20260619 02:54:53.609176 3583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:53.608463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.2:41401)
0619 02:54:53.608917 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:53.609004 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":210,"threads_started":1}
I20260619 02:54:54.039096 3584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:54.038368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.3:47735)
0619 02:54:54.038833 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:54.038922 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":207,"threads_started":1}
I20260619 02:54:54.187434 3585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:54.186583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.1:38343)
0619 02:54:54.187116 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:54.187209 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":295,"threads_started":1}
I20260619 02:54:54.611054 3586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:54.610202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.2:44597)
0619 02:54:54.610741 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:54.610828 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":285,"threads_started":1}
I20260619 02:54:55.040791 3587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:55.040034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.3:46615)
0619 02:54:55.040518 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:55.040601 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":249,"threads_started":1}
I20260619 02:54:55.189278 3588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:55.188497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.1:50985)
0619 02:54:55.188981 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:55.189094 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":244,"threads_started":1}
I20260619 02:54:55.612675 3589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:55.611922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.2:57979)
0619 02:54:55.612404 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:55.612493 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":217,"threads_started":1}
I20260619 02:54:56.042412 3590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:56.041690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.3:60869)
0619 02:54:56.042136 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:56.042221 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":214,"threads_started":1}
I20260619 02:54:56.190923 3591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:56.190187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.1:56739)
0619 02:54:56.190664 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:56.190748 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":236,"threads_started":1}
I20260619 02:54:56.614311 3592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:56.613607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.2:37533)
0619 02:54:56.614062 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:56.614145 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":226,"threads_started":1}
I20260619 02:54:57.044118 3593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:57.043394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.3:53943)
0619 02:54:57.043872 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:57.043964 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":218,"threads_started":1}
I20260619 02:54:57.192553 3594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:57.191820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.1:38925)
0619 02:54:57.192269 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:57.192382 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":232,"threads_started":1}
I20260619 02:54:57.615976 3595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:57.615189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.2:57621)
0619 02:54:57.615684 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:57.615803 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":234,"threads_started":1}
I20260619 02:54:58.045975 3596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:58.045092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.3:52145)
0619 02:54:58.045611 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:58.045738 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":242,"threads_started":1}
I20260619 02:54:58.194252 3597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:58.193533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.1:34153)
0619 02:54:58.193989 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:58.194075 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":236,"threads_started":1}
I20260619 02:54:58.617689 3598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:58.616924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.2:34855)
0619 02:54:58.617430 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:58.617513 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":228,"threads_started":1}
I20260619 02:54:59.047571 3599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:59.046869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.3:55789)
0619 02:54:59.047307 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:59.047396 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":208,"threads_started":1}
I20260619 02:54:59.196135 3600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:59.195420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.1:39309)
0619 02:54:59.195863 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:59.195955 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":215,"threads_started":1}
I20260619 02:54:59.619660 3601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:54:59.618880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.2:57029)
0619 02:54:59.619372 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0619 02:54:59.619477 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":264,"threads_started":1}
I20260619 02:55:00.023629 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 3124
I20260619 02:55:00.042254 3602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:00.024811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52126 (local address 127.18.134.1:41291)
0619 02:55:00.025329 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:00.025343 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:00.025381 (+ 38us) server_negotiation.cc:407] Connection header received
0619 02:55:00.026915 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:00.026925 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:00.027345 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:00.027628 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:00.029031 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:00.030265 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:00.031594 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:00.032289 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:00.032431 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:00.035362 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:00.035373 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:00.035392 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:00.035440 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:00.038614 (+ 3174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:00.039513 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:00.039522 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:00.039531 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:00.039757 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:00.040176 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:00.040189 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:00.040192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:00.040914 (+ 722us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:00.041265 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:00.041615 (+ 350us) server_negotiation.cc:299] Negotiation successful
0619 02:55:00.041964 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":232,"threads_started":1}
I20260619 02:55:00.044339 3287 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:00.049289 3604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:00.048566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35511 (local address 127.18.134.3:55841)
0619 02:55:00.049027 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:00.049127 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.62:35511: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":239,"threads_started":1}
I20260619 02:55:00.126498 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 3124
I20260619 02:55:00.140501 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 3295
I20260619 02:55:00.161639 3605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:00.141451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57156 (local address 127.18.134.2:37295)
0619 02:55:00.141988 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:00.142001 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:00.142038 (+ 37us) server_negotiation.cc:407] Connection header received
0619 02:55:00.143673 (+ 1635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:00.143682 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:00.144090 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:00.144375 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:00.145733 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:00.147007 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:00.148348 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:00.149026 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:00.149157 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:00.155621 (+ 6464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:00.155631 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:00.155649 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:00.155699 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:00.158591 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:00.159293 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:00.159300 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:00.159306 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:00.159478 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:00.159814 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:00.159818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:00.159820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:00.160338 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:00.160565 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:00.160959 (+ 394us) server_negotiation.cc:299] Negotiation successful
0619 02:55:00.161361 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":238,"threads_started":1}
I20260619 02:55:00.163028 3418 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:00.249306 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 3295
I20260619 02:55:00.263118 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 3426
I20260619 02:55:00.280854 3607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:00.264081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36944 (local address 127.18.134.3:43255)
0619 02:55:00.264709 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:00.264723 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:00.264764 (+ 41us) server_negotiation.cc:407] Connection header received
0619 02:55:00.266427 (+ 1663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:00.266437 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:00.266982 (+ 545us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:00.267276 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:00.268540 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:00.269750 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:00.271046 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:00.271797 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:00.271956 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:00.274561 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:00.274579 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:00.274594 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:00.274661 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:00.277582 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:00.278244 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:00.278250 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:00.278257 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:00.278431 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:00.278925 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:00.278929 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:00.278931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:00.279510 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:00.279761 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:00.280130 (+ 369us) server_negotiation.cc:299] Negotiation successful
0619 02:55:00.280478 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":249,"threads_started":1}
I20260619 02:55:00.282402 3549 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:00.364012 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 3426
2026-06-19T02:55:00Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (63862 ms)
[----------] 30 tests from SecurityITest (289806 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:55:00 dist-test-slave-9mr0 krb5kdc[3611](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:55:00 dist-test-slave-9mr0 krb5kdc[3611](info): set up 2 sockets
Jun 19 02:55:00 dist-test-slave-9mr0 krb5kdc[3611](info): commencing operation
krb5kdc: starting...
W20260619 02:55:01.378696 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.979s user 0.002s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:55:01 dist-test-slave-9mr0 krb5kdc[3611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:55:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:55:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:55:01.614925 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:44503
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:34747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:44503
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:01.931382 3624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:01.931686 3624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:01.931805 3624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:01.939225 3624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:01.939337 3624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:01.939394 3624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:01.939471 3624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:01.939518 3624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:01.949856 3624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:34747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:44503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:44503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:01.951244 3624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:01.952528 3624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:01.964373 3629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:01.964457 3630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:01.964399 3632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:01.964737 3624 server_base.cc:1034] running on GCE node
I20260619 02:55:01.965660 3624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:01.967093 3624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:01.968302 3624 hybrid_clock.cc:648] HybridClock initialized: now 1781837701968247 us; error 33 us; skew 500 ppm
I20260619 02:55:01.968668 3624 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:01 dist-test-slave-9mr0 krb5kdc[3611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837701, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:01.971865 3624 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:55:01.973289 3624 webserver.cc:460] Webserver started at http://127.18.134.62:33895/ using document root <none> and password file <none>
I20260619 02:55:01.973933 3624 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:01.974030 3624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:01.974280 3624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:01.977222 3624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "c3ff00b9099c4af7ac3d9621cefd4d4e"
format_stamp: "Formatted at 2026-06-19 02:55:01 on dist-test-slave-9mr0"
server_key: "f68ef7adf7858550a977e194e3ed48ff"
server_key_iv: "41b807d3552e3e6ce22d5a58117c00af"
server_key_version: "encryptionkey@0"
I20260619 02:55:01.977937 3624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "c3ff00b9099c4af7ac3d9621cefd4d4e"
format_stamp: "Formatted at 2026-06-19 02:55:01 on dist-test-slave-9mr0"
server_key: "f68ef7adf7858550a977e194e3ed48ff"
server_key_iv: "41b807d3552e3e6ce22d5a58117c00af"
server_key_version: "encryptionkey@0"
I20260619 02:55:01.983392 3624 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 02:55:01.986894 3639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:01.988245 3624 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 02:55:01.988405 3624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "c3ff00b9099c4af7ac3d9621cefd4d4e"
format_stamp: "Formatted at 2026-06-19 02:55:01 on dist-test-slave-9mr0"
server_key: "f68ef7adf7858550a977e194e3ed48ff"
server_key_iv: "41b807d3552e3e6ce22d5a58117c00af"
server_key_version: "encryptionkey@0"
I20260619 02:55:01.988587 3624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:02.011914 3624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:02.016193 3624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:02.016485 3624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:02.034224 3624 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:44503
I20260619 02:55:02.034299 3690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:44503 every 8 connection(s)
I20260619 02:55:02.035585 3624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:55:02.037007 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 3624
I20260619 02:55:02.037228 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:55:02.038043 18968 external_mini_cluster.cc:1428] Setting key dca4dd87ddafaf7a835dcbbec9c762d5
I20260619 02:55:02.040162 3691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:02.051038 3691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3ff00b9099c4af7ac3d9621cefd4d4e: Bootstrap starting.
Jun 19 02:55:02 dist-test-slave-9mr0 krb5kdc[3611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:55:02.054119 3691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3ff00b9099c4af7ac3d9621cefd4d4e: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:02.055198 3691 log.cc:824] T 00000000000000000000000000000000 P c3ff00b9099c4af7ac3d9621cefd4d4e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:02.057879 3691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3ff00b9099c4af7ac3d9621cefd4d4e: No bootstrap required, opened a new log
I20260619 02:55:02.059535 3694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:02.040159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37042 (local address 127.18.134.62:44503)
0619 02:55:02.040909 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:02.040920 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:02.040961 (+ 41us) server_negotiation.cc:407] Connection header received
0619 02:55:02.042591 (+ 1630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:02.042613 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:02.043040 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:02.043427 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:02.044838 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:02.046174 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:02.047550 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:02.048234 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:02.048425 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:02.052621 (+ 4196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:02.052629 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:02.052642 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:02.052686 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:02.055725 (+ 3039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:02.056393 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:02.056399 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:02.056405 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:02.056567 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:02.056935 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:02.056938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:02.056940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:02.057438 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:02.057729 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:02.058125 (+ 396us) server_negotiation.cc:299] Negotiation successful
0619 02:55:02.058531 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":319,"threads_started":1}
I20260619 02:55:02.065537 3691 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3ff00b9099c4af7ac3d9621cefd4d4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3ff00b9099c4af7ac3d9621cefd4d4e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 44503 } }
I20260619 02:55:02.066045 3691 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3ff00b9099c4af7ac3d9621cefd4d4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:02.066154 3691 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3ff00b9099c4af7ac3d9621cefd4d4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3ff00b9099c4af7ac3d9621cefd4d4e, State: Initialized, Role: FOLLOWER
I20260619 02:55:02.066900 3691 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3ff00b9099c4af7ac3d9621cefd4d4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3ff00b9099c4af7ac3d9621cefd4d4e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 44503 } }
I20260619 02:55:02.067122 3691 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3ff00b9099c4af7ac3d9621cefd4d4e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:55:02.067262 3691 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3ff00b9099c4af7ac3d9621cefd4d4e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:55:02.067407 3691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3ff00b9099c4af7ac3d9621cefd4d4e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:02.069571 3691 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3ff00b9099c4af7ac3d9621cefd4d4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3ff00b9099c4af7ac3d9621cefd4d4e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 44503 } }
I20260619 02:55:02.070072 3691 leader_election.cc:302] T 00000000000000000000000000000000 P c3ff00b9099c4af7ac3d9621cefd4d4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3ff00b9099c4af7ac3d9621cefd4d4e; no voters:
I20260619 02:55:02.070611 3691 leader_election.cc:288] T 00000000000000000000000000000000 P c3ff00b9099c4af7ac3d9621cefd4d4e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:55:02.070778 3696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3ff00b9099c4af7ac3d9621cefd4d4e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:02.071784 3696 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3ff00b9099c4af7ac3d9621cefd4d4e [term 1 LEADER]: Becoming Leader. State: Replica: c3ff00b9099c4af7ac3d9621cefd4d4e, State: Running, Role: LEADER
I20260619 02:55:02.072444 3696 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3ff00b9099c4af7ac3d9621cefd4d4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3ff00b9099c4af7ac3d9621cefd4d4e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 44503 } }
I20260619 02:55:02.072649 3691 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3ff00b9099c4af7ac3d9621cefd4d4e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:55:02.075650 3698 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3ff00b9099c4af7ac3d9621cefd4d4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3ff00b9099c4af7ac3d9621cefd4d4e. Latest consensus state: current_term: 1 leader_uuid: "c3ff00b9099c4af7ac3d9621cefd4d4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3ff00b9099c4af7ac3d9621cefd4d4e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 44503 } } }
I20260619 02:55:02.076020 3698 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3ff00b9099c4af7ac3d9621cefd4d4e [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:02.076485 3697 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3ff00b9099c4af7ac3d9621cefd4d4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3ff00b9099c4af7ac3d9621cefd4d4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3ff00b9099c4af7ac3d9621cefd4d4e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 44503 } } }
I20260619 02:55:02.076782 3697 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3ff00b9099c4af7ac3d9621cefd4d4e [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:02.079372 3705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:55:02.083811 3705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:55:02.091521 3705 catalog_manager.cc:1347] Generated new cluster ID: 30efb8cc048d4f669b715474036cad79
I20260619 02:55:02.091639 3705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:55:02.114395 3705 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:55:02.115765 3705 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:55:02.128571 3705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3ff00b9099c4af7ac3d9621cefd4d4e: Generated new TSK 0
I20260619 02:55:02.129442 3705 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.134.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
I20260619 02:55:02.189734 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:0
--local_ip_for_outbound_sockets=127.18.134.1
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:44503
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:34747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:02.508394 3717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:02.508699 3717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:02.508859 3717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:02.516415 3717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:02.516552 3717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:02.516714 3717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:55:02.526793 3717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:34747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:44503
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:02.528074 3717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:02.529363 3717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:02.542493 3725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:02.542500 3723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:02.543045 3722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:02.543123 3717 server_base.cc:1034] running on GCE node
I20260619 02:55:02.543813 3717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:02.544735 3717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:02.545935 3717 hybrid_clock.cc:648] HybridClock initialized: now 1781837702545878 us; error 49 us; skew 500 ppm
I20260619 02:55:02.546247 3717 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:02 dist-test-slave-9mr0 krb5kdc[3611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837702, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:02.549263 3717 init.cc:377] Logged in from keytab as kudu/127.18.134.1@KRBTEST.COM (short username kudu)
I20260619 02:55:02.550784 3717 webserver.cc:460] Webserver started at http://127.18.134.1:40021/ using document root <none> and password file <none>
I20260619 02:55:02.551491 3717 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:02.551687 3717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:02.552023 3717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:02.555033 3717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-0/data/instance:
uuid: "32138a8519284a09b45a3a01d8e008fe"
format_stamp: "Formatted at 2026-06-19 02:55:02 on dist-test-slave-9mr0"
server_key: "e0923172117b58d56d5bda108a75744f"
server_key_iv: "5093c0905fd7370a408031adf1b001cc"
server_key_version: "encryptionkey@0"
I20260619 02:55:02.555765 3717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance:
uuid: "32138a8519284a09b45a3a01d8e008fe"
format_stamp: "Formatted at 2026-06-19 02:55:02 on dist-test-slave-9mr0"
server_key: "e0923172117b58d56d5bda108a75744f"
server_key_iv: "5093c0905fd7370a408031adf1b001cc"
server_key_version: "encryptionkey@0"
I20260619 02:55:02.561060 3717 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260619 02:55:02.564484 3732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:02.565829 3717 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:55:02.565982 3717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "32138a8519284a09b45a3a01d8e008fe"
format_stamp: "Formatted at 2026-06-19 02:55:02 on dist-test-slave-9mr0"
server_key: "e0923172117b58d56d5bda108a75744f"
server_key_iv: "5093c0905fd7370a408031adf1b001cc"
server_key_version: "encryptionkey@0"
I20260619 02:55:02.566167 3717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:02.581579 3717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:02.585630 3717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:02.585928 3717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:02.587059 3717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:55:02.588600 3717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:55:02.588681 3717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:02.588816 3717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:55:02.588871 3717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:02.619673 3717 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:35707
I20260619 02:55:02.619738 3844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:35707 every 8 connection(s)
I20260619 02:55:02.620945 3717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:55:02.625617 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 3717
I20260619 02:55:02.625818 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance
I20260619 02:55:02.626564 18968 external_mini_cluster.cc:1428] Setting key cab81b583b5172ff4771f03aa05f5e65
Jun 19 02:55:02 dist-test-slave-9mr0 krb5kdc[3611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837702, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:55:02.641757 3847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:02.623438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:45023 (local address 127.18.134.62:44503)
0619 02:55:02.623865 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:02.623869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:02.624744 (+ 875us) server_negotiation.cc:407] Connection header received
0619 02:55:02.627107 (+ 2363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:02.627111 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:02.627194 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:02.627331 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:02.629443 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:02.630447 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:02.632166 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:02.632706 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:02.632817 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:02.636269 (+ 3452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:02.636274 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:02.636277 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:02.636312 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:02.638765 (+ 2453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:02.639613 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:02.639616 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:02.639619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:02.639732 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:02.640149 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:02.640152 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:02.640154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:02.640417 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:02.640522 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:02.641335 (+ 813us) server_negotiation.cc:299] Negotiation successful
0619 02:55:02.641503 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":201,"threads_started":1}
I20260619 02:55:02.643075 3848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:02.623940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:44503 (local address 127.18.134.1:45023)
0619 02:55:02.624590 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:02.624627 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:02.626800 (+ 2173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:02.627602 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:02.627611 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:02.628353 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:02.629254 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:02.629273 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:02.630625 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:02.630653 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:02.632022 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:02.632035 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:02.632908 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:02.632912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:02.633105 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:02.633762 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:55:02.633796 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:02.636036 (+ 2240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:02.638930 (+ 2894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:02.638939 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:02.638955 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:02.639506 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:02.639823 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:02.639828 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:02.639831 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:02.640056 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:02.640524 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:02.640531 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:02.640949 (+ 418us) client_negotiation.cc:769] Sending connection context
0619 02:55:02.641217 (+ 268us) client_negotiation.cc:240] Negotiation successful
0619 02:55:02.641737 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":240,"threads_started":1}
I20260619 02:55:02.644572 3845 heartbeater.cc:347] Connected to a master server at 127.18.134.62:44503
I20260619 02:55:02.644853 3845 heartbeater.cc:464] Registering TS with master...
I20260619 02:55:02.645535 3845 heartbeater.cc:511] Master 127.18.134.62:44503 requested a full tablet report, sending...
I20260619 02:55:02.647795 3656 ts_manager.cc:194] Registered new tserver with Master: 32138a8519284a09b45a3a01d8e008fe (127.18.134.1:35707)
I20260619 02:55:02.649832 3656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.1@KRBTEST.COM'} at 127.18.134.1:45023
WARNING: no policy specified for kudu/127.18.134.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
I20260619 02:55:02.679751 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:0
--local_ip_for_outbound_sockets=127.18.134.2
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:44503
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:34747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:02.989126 3851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:02.989403 3851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:02.989529 3851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:02.996847 3851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:02.996981 3851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:02.997140 3851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:55:03.006886 3851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:34747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:44503
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:03.008332 3851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:03.009639 3851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:03.022475 3859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:03.022533 3857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:03.022500 3856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:03.023110 3851 server_base.cc:1034] running on GCE node
I20260619 02:55:03.023759 3851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:03.024680 3851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:03.025895 3851 hybrid_clock.cc:648] HybridClock initialized: now 1781837703025844 us; error 44 us; skew 500 ppm
I20260619 02:55:03.026211 3851 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:03 dist-test-slave-9mr0 krb5kdc[3611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837703, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:03.029350 3851 init.cc:377] Logged in from keytab as kudu/127.18.134.2@KRBTEST.COM (short username kudu)
I20260619 02:55:03.030874 3851 webserver.cc:460] Webserver started at http://127.18.134.2:35579/ using document root <none> and password file <none>
I20260619 02:55:03.031548 3851 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:03.031651 3851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:03.031965 3851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:03.035006 3851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-1/data/instance:
uuid: "c0dbed1d5c444bdfb9b6abf847d57eb3"
format_stamp: "Formatted at 2026-06-19 02:55:03 on dist-test-slave-9mr0"
server_key: "9580718713be5d32f007468ca0353bc8"
server_key_iv: "152387f30993fdf5e35b29c14e43ab4a"
server_key_version: "encryptionkey@0"
I20260619 02:55:03.035754 3851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance:
uuid: "c0dbed1d5c444bdfb9b6abf847d57eb3"
format_stamp: "Formatted at 2026-06-19 02:55:03 on dist-test-slave-9mr0"
server_key: "9580718713be5d32f007468ca0353bc8"
server_key_iv: "152387f30993fdf5e35b29c14e43ab4a"
server_key_version: "encryptionkey@0"
I20260619 02:55:03.041086 3851 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:55:03.044647 3866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:03.046034 3851 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:55:03.046186 3851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "c0dbed1d5c444bdfb9b6abf847d57eb3"
format_stamp: "Formatted at 2026-06-19 02:55:03 on dist-test-slave-9mr0"
server_key: "9580718713be5d32f007468ca0353bc8"
server_key_iv: "152387f30993fdf5e35b29c14e43ab4a"
server_key_version: "encryptionkey@0"
I20260619 02:55:03.046383 3851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:03.073355 3851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:03.077605 3851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:03.077858 3851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:03.078804 3851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:55:03.080127 3851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:55:03.080200 3851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:03.080281 3851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:55:03.080324 3851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:03.109475 3851 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:41173
I20260619 02:55:03.109544 3978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:41173 every 8 connection(s)
I20260619 02:55:03.110710 3851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:55:03.120625 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 3851
I20260619 02:55:03.120816 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance
I20260619 02:55:03.121325 18968 external_mini_cluster.cc:1428] Setting key bfaa5bad39947718da2d6ca68a1f11e2
Jun 19 02:55:03 dist-test-slave-9mr0 krb5kdc[3611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837703, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:55:03.129523 3847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:03.113049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:36009 (local address 127.18.134.62:44503)
0619 02:55:03.113199 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:03.113203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:03.114408 (+ 1205us) server_negotiation.cc:407] Connection header received
0619 02:55:03.116138 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:03.116142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:03.116215 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:03.116304 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:03.117655 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.118368 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.119845 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.120228 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.120317 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:03.124057 (+ 3740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:03.124063 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:03.124066 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:03.124097 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:03.126548 (+ 2451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:03.127269 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:03.127273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:03.127275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:03.127396 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:03.127792 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:03.127795 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:03.127797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:03.128051 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:03.128178 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:03.128919 (+ 741us) server_negotiation.cc:299] Negotiation successful
0619 02:55:03.129161 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 02:55:03.130285 3981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:03.113561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:44503 (local address 127.18.134.2:36009)
0619 02:55:03.114238 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:03.114285 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:03.115897 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:03.116476 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:03.116483 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:03.116994 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:03.117521 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:03.117538 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.118488 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.118492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:03.119721 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:03.119733 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.120439 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.120446 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:03.120739 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:03.121413 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:55:03.121442 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:03.123842 (+ 2400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:03.126704 (+ 2862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:03.126713 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:03.126731 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:03.127141 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:03.127496 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:03.127501 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:03.127505 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:03.127682 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:03.128171 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:03.128179 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:03.128499 (+ 320us) client_negotiation.cc:769] Sending connection context
0619 02:55:03.128774 (+ 275us) client_negotiation.cc:240] Negotiation successful
0619 02:55:03.129236 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":248,"threads_started":1}
I20260619 02:55:03.131659 3979 heartbeater.cc:347] Connected to a master server at 127.18.134.62:44503
I20260619 02:55:03.131951 3979 heartbeater.cc:464] Registering TS with master...
I20260619 02:55:03.132673 3979 heartbeater.cc:511] Master 127.18.134.62:44503 requested a full tablet report, sending...
I20260619 02:55:03.134048 3656 ts_manager.cc:194] Registered new tserver with Master: c0dbed1d5c444bdfb9b6abf847d57eb3 (127.18.134.2:41173)
I20260619 02:55:03.134879 3656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.2@KRBTEST.COM'} at 127.18.134.2:36009
WARNING: no policy specified for kudu/127.18.134.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
I20260619 02:55:03.169667 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:0
--local_ip_for_outbound_sockets=127.18.134.3
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:44503
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:34747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:03.486611 3984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:03.486928 3984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:03.487051 3984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:03.494222 3984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:03.494338 3984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:03.494496 3984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:55:03.504155 3984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:34747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:44503
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:03.505378 3984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:03.506589 3984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:03.518883 3992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:03.518910 3990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:03.519176 3989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:03.519624 3984 server_base.cc:1034] running on GCE node
I20260619 02:55:03.520215 3984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:03.521066 3984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:03.522259 3984 hybrid_clock.cc:648] HybridClock initialized: now 1781837703522210 us; error 40 us; skew 500 ppm
I20260619 02:55:03.522555 3984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:03 dist-test-slave-9mr0 krb5kdc[3611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837703, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:03.525622 3984 init.cc:377] Logged in from keytab as kudu/127.18.134.3@KRBTEST.COM (short username kudu)
I20260619 02:55:03.527197 3984 webserver.cc:460] Webserver started at http://127.18.134.3:36921/ using document root <none> and password file <none>
I20260619 02:55:03.527856 3984 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:03.527971 3984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:03.528254 3984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:03.531116 3984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-2/data/instance:
uuid: "eb536155322d4e1d9b62ca5ae95553be"
format_stamp: "Formatted at 2026-06-19 02:55:03 on dist-test-slave-9mr0"
server_key: "bc09d362dc818d8f939d43b72db0a349"
server_key_iv: "72ef796f7509df7d11d1f06411a63be4"
server_key_version: "encryptionkey@0"
I20260619 02:55:03.531821 3984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance:
uuid: "eb536155322d4e1d9b62ca5ae95553be"
format_stamp: "Formatted at 2026-06-19 02:55:03 on dist-test-slave-9mr0"
server_key: "bc09d362dc818d8f939d43b72db0a349"
server_key_iv: "72ef796f7509df7d11d1f06411a63be4"
server_key_version: "encryptionkey@0"
I20260619 02:55:03.536713 3984 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:55:03.540095 3999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:03.541419 3984 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:55:03.541576 3984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "eb536155322d4e1d9b62ca5ae95553be"
format_stamp: "Formatted at 2026-06-19 02:55:03 on dist-test-slave-9mr0"
server_key: "bc09d362dc818d8f939d43b72db0a349"
server_key_iv: "72ef796f7509df7d11d1f06411a63be4"
server_key_version: "encryptionkey@0"
I20260619 02:55:03.541757 3984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:03.576566 3984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:03.580804 3984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:03.581071 3984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:03.582087 3984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:55:03.583521 3984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:55:03.583606 3984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:03.583734 3984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:55:03.583787 3984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:03.614197 3984 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:42743
I20260619 02:55:03.614243 4111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:42743 every 8 connection(s)
I20260619 02:55:03.615465 3984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:55:03.621783 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 3984
I20260619 02:55:03.621978 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance
I20260619 02:55:03.622514 18968 external_mini_cluster.cc:1428] Setting key 9623f948f6aba7a5b9b7699d079a8963
Jun 19 02:55:03 dist-test-slave-9mr0 krb5kdc[3611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837703, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:55:03.635092 3847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:03.617907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:38337 (local address 127.18.134.62:44503)
0619 02:55:03.618075 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:03.618079 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:03.619213 (+ 1134us) server_negotiation.cc:407] Connection header received
0619 02:55:03.621027 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:03.621032 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:03.621111 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:03.621198 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:03.623258 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.624409 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.626122 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.626524 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.626622 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:03.629879 (+ 3257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:03.629884 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:03.629888 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:03.629921 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:03.632345 (+ 2424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:03.633010 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:03.633014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:03.633016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:03.633127 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:03.633544 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:03.633547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:03.633549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:03.633821 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:03.633929 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:03.634615 (+ 686us) server_negotiation.cc:299] Negotiation successful
0619 02:55:03.634837 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 02:55:03.636035 4114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:03.618360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:44503 (local address 127.18.134.3:38337)
0619 02:55:03.619053 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:03.619089 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:03.620767 (+ 1678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:03.621454 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:03.621465 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:03.622185 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:03.623088 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:03.623109 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.624604 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.624627 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:03.625997 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:03.626010 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.626759 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.626764 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:03.626949 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:03.627588 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:55:03.627615 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:03.629679 (+ 2064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:03.632472 (+ 2793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:03.632479 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:03.632492 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:03.632915 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:03.633252 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:03.633255 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:03.633258 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:03.633452 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:03.633957 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:03.633963 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:03.634310 (+ 347us) client_negotiation.cc:769] Sending connection context
0619 02:55:03.634535 (+ 225us) client_negotiation.cc:240] Negotiation successful
0619 02:55:03.634962 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":289,"threads_started":1}
I20260619 02:55:03.637324 4112 heartbeater.cc:347] Connected to a master server at 127.18.134.62:44503
I20260619 02:55:03.637681 4112 heartbeater.cc:464] Registering TS with master...
I20260619 02:55:03.638451 4112 heartbeater.cc:511] Master 127.18.134.62:44503 requested a full tablet report, sending...
I20260619 02:55:03.639835 3655 ts_manager.cc:194] Registered new tserver with Master: eb536155322d4e1d9b62ca5ae95553be (127.18.134.3:42743)
I20260619 02:55:03.640671 3655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.3@KRBTEST.COM'} at 127.18.134.3:38337
I20260619 02:55:03.650540 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:55:03.652966 3845 heartbeater.cc:503] Master 127.18.134.62:44503 was elected leader, sending a full tablet report...
I20260619 02:55:03.667236 3847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:03.655196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37044 (local address 127.18.134.62:44503)
0619 02:55:03.655422 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:03.655428 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:03.655654 (+ 226us) server_negotiation.cc:407] Connection header received
0619 02:55:03.655775 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:03.655779 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:03.655838 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:03.655915 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:03.657450 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.658146 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.659619 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.660013 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.660103 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:03.662283 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:03.662289 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:03.662292 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:03.662324 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:03.664732 (+ 2408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:03.665390 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:03.665394 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:03.665396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:03.665526 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:03.665922 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:03.665925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:03.665927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:03.666220 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:03.666411 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:03.666819 (+ 408us) server_negotiation.cc:299] Negotiation successful
0619 02:55:03.666985 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260619 02:55:03.671738 3655 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37044:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:55:03.678812 3655 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:55:03.713594 4126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:03.691615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38422 (local address 127.18.134.3:42743)
0619 02:55:03.703544 (+ 11929us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:03.703554 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:03.703591 (+ 37us) server_negotiation.cc:407] Connection header received
0619 02:55:03.703697 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:03.703704 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:03.705031 (+ 1327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:03.705251 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:03.706290 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.708372 (+ 2082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.710741 (+ 2369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.712005 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.712137 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:03.712372 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:03.712901 (+ 529us) server_negotiation.cc:299] Negotiation successful
0619 02:55:03.713156 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11758,"thread_start_us":11599,"threads_started":1}
I20260619 02:55:03.714397 4125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:03.691367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:42743 (local address 127.0.0.1:38422)
0619 02:55:03.702561 (+ 11194us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:03.702619 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:03.702874 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:03.705591 (+ 2717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:03.705600 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:03.705635 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:03.706033 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:03.706047 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.708502 (+ 2455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.708507 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:03.710535 (+ 2028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:03.710547 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.712448 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.712455 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:03.712649 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:03.712705 (+ 56us) client_negotiation.cc:769] Sending connection context
0619 02:55:03.712839 (+ 134us) client_negotiation.cc:240] Negotiation successful
0619 02:55:03.713970 (+ 1131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10845,"mutex_wait_us":67,"thread_start_us":12425,"threads_started":1}
I20260619 02:55:03.714495 4129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:03.704294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:41173 (local address 127.0.0.1:58366)
0619 02:55:03.706498 (+ 2204us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:03.706530 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:03.706675 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:03.707518 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:03.707523 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:03.707547 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:03.707946 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:03.707955 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.709750 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.709755 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:03.711967 (+ 2212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:03.711980 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.713768 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.713774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:03.713931 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:03.713967 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 02:55:03.714051 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 02:55:03.714165 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2075,"mutex_wait_us":65,"thread_start_us":1910,"threads_started":1}
I20260619 02:55:03.714859 4124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:03.691570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:35707 (local address 127.0.0.1:44688)
0619 02:55:03.703926 (+ 12356us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:03.703944 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:03.704065 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:03.707541 (+ 3476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:03.707546 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:03.707574 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:03.708096 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:03.708108 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.709618 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.709621 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:03.712245 (+ 2624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:03.712255 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.713977 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.713983 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:03.714139 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:03.714172 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 02:55:03.714238 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 02:55:03.714376 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12079,"mutex_wait_us":107,"thread_start_us":10469,"threads_started":1}
I20260619 02:55:03.715548 4128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:03.703442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58366 (local address 127.18.134.2:41173)
0619 02:55:03.706074 (+ 2632us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:03.706084 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:03.706796 (+ 712us) server_negotiation.cc:407] Connection header received
0619 02:55:03.706879 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:03.706885 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:03.707113 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:03.707295 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:03.708160 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.709610 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.712116 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.713301 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.713441 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:03.713634 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:03.715103 (+ 1469us) server_negotiation.cc:299] Negotiation successful
0619 02:55:03.715276 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2466,"thread_start_us":2363,"threads_started":1}
I20260619 02:55:03.715801 4127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:03.691979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44688 (local address 127.18.134.1:35707)
0619 02:55:03.706922 (+ 14943us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:03.706930 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:03.707009 (+ 79us) server_negotiation.cc:407] Connection header received
0619 02:55:03.707106 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:03.707114 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:03.707333 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:03.708287 (+ 954us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:03.708537 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.709486 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.712396 (+ 2910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.713728 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.714487 (+ 759us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:03.715269 (+ 782us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:03.715336 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 02:55:03.715484 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14727,"thread_start_us":14540,"threads_started":1}
I20260619 02:55:03.718796 3780 tablet_service.cc:1467] Processing CreateTablet for tablet abffc195a4b24d7db9864ba5f3979d3a (DEFAULT_TABLE table=test-table [id=a2e364501b734d3080a1de89df4ddac3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:55:03.718797 4047 tablet_service.cc:1467] Processing CreateTablet for tablet abffc195a4b24d7db9864ba5f3979d3a (DEFAULT_TABLE table=test-table [id=a2e364501b734d3080a1de89df4ddac3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:55:03.718797 3914 tablet_service.cc:1467] Processing CreateTablet for tablet abffc195a4b24d7db9864ba5f3979d3a (DEFAULT_TABLE table=test-table [id=a2e364501b734d3080a1de89df4ddac3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:55:03.720007 3780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abffc195a4b24d7db9864ba5f3979d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:03.720108 4047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abffc195a4b24d7db9864ba5f3979d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:03.720119 3914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abffc195a4b24d7db9864ba5f3979d3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:03.730976 4130 tablet_bootstrap.cc:492] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe: Bootstrap starting.
I20260619 02:55:03.733807 4132 tablet_bootstrap.cc:492] T abffc195a4b24d7db9864ba5f3979d3a P c0dbed1d5c444bdfb9b6abf847d57eb3: Bootstrap starting.
I20260619 02:55:03.734836 4131 tablet_bootstrap.cc:492] T abffc195a4b24d7db9864ba5f3979d3a P eb536155322d4e1d9b62ca5ae95553be: Bootstrap starting.
I20260619 02:55:03.735039 4130 tablet_bootstrap.cc:654] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:03.736302 4130 log.cc:824] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:03.737977 4132 tablet_bootstrap.cc:654] T abffc195a4b24d7db9864ba5f3979d3a P c0dbed1d5c444bdfb9b6abf847d57eb3: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:03.738034 4131 tablet_bootstrap.cc:654] T abffc195a4b24d7db9864ba5f3979d3a P eb536155322d4e1d9b62ca5ae95553be: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:03.739054 4132 log.cc:824] T abffc195a4b24d7db9864ba5f3979d3a P c0dbed1d5c444bdfb9b6abf847d57eb3: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:03.739279 4131 log.cc:824] T abffc195a4b24d7db9864ba5f3979d3a P eb536155322d4e1d9b62ca5ae95553be: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:03.739490 4130 tablet_bootstrap.cc:492] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe: No bootstrap required, opened a new log
I20260619 02:55:03.739763 4130 ts_tablet_manager.cc:1397] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260619 02:55:03.742129 4131 tablet_bootstrap.cc:492] T abffc195a4b24d7db9864ba5f3979d3a P eb536155322d4e1d9b62ca5ae95553be: No bootstrap required, opened a new log
I20260619 02:55:03.742245 4132 tablet_bootstrap.cc:492] T abffc195a4b24d7db9864ba5f3979d3a P c0dbed1d5c444bdfb9b6abf847d57eb3: No bootstrap required, opened a new log
I20260619 02:55:03.742411 4131 ts_tablet_manager.cc:1397] T abffc195a4b24d7db9864ba5f3979d3a P eb536155322d4e1d9b62ca5ae95553be: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260619 02:55:03.742523 4132 ts_tablet_manager.cc:1397] T abffc195a4b24d7db9864ba5f3979d3a P c0dbed1d5c444bdfb9b6abf847d57eb3: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260619 02:55:03.747722 4130 raft_consensus.cc:348] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb536155322d4e1d9b62ca5ae95553be" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42743 } } peers { permanent_uuid: "32138a8519284a09b45a3a01d8e008fe" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 35707 } } peers { permanent_uuid: "c0dbed1d5c444bdfb9b6abf847d57eb3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 41173 } }
I20260619 02:55:03.748080 4130 raft_consensus.cc:374] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:03.748210 4130 raft_consensus.cc:729] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32138a8519284a09b45a3a01d8e008fe, State: Initialized, Role: FOLLOWER
I20260619 02:55:03.748791 4130 consensus_queue.cc:260] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb536155322d4e1d9b62ca5ae95553be" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42743 } } peers { permanent_uuid: "32138a8519284a09b45a3a01d8e008fe" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 35707 } } peers { permanent_uuid: "c0dbed1d5c444bdfb9b6abf847d57eb3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 41173 } }
I20260619 02:55:03.750571 4130 ts_tablet_manager.cc:1428] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe: Time spent starting tablet: real 0.011s user 0.009s sys 0.001s
I20260619 02:55:03.752698 4131 raft_consensus.cc:348] T abffc195a4b24d7db9864ba5f3979d3a P eb536155322d4e1d9b62ca5ae95553be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb536155322d4e1d9b62ca5ae95553be" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42743 } } peers { permanent_uuid: "32138a8519284a09b45a3a01d8e008fe" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 35707 } } peers { permanent_uuid: "c0dbed1d5c444bdfb9b6abf847d57eb3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 41173 } }
I20260619 02:55:03.752696 4132 raft_consensus.cc:348] T abffc195a4b24d7db9864ba5f3979d3a P c0dbed1d5c444bdfb9b6abf847d57eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb536155322d4e1d9b62ca5ae95553be" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42743 } } peers { permanent_uuid: "32138a8519284a09b45a3a01d8e008fe" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 35707 } } peers { permanent_uuid: "c0dbed1d5c444bdfb9b6abf847d57eb3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 41173 } }
I20260619 02:55:03.753100 4132 raft_consensus.cc:374] T abffc195a4b24d7db9864ba5f3979d3a P c0dbed1d5c444bdfb9b6abf847d57eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:03.753113 4131 raft_consensus.cc:374] T abffc195a4b24d7db9864ba5f3979d3a P eb536155322d4e1d9b62ca5ae95553be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:03.753209 4131 raft_consensus.cc:729] T abffc195a4b24d7db9864ba5f3979d3a P eb536155322d4e1d9b62ca5ae95553be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb536155322d4e1d9b62ca5ae95553be, State: Initialized, Role: FOLLOWER
I20260619 02:55:03.753203 4132 raft_consensus.cc:729] T abffc195a4b24d7db9864ba5f3979d3a P c0dbed1d5c444bdfb9b6abf847d57eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0dbed1d5c444bdfb9b6abf847d57eb3, State: Initialized, Role: FOLLOWER
I20260619 02:55:03.753769 4131 consensus_queue.cc:260] T abffc195a4b24d7db9864ba5f3979d3a P eb536155322d4e1d9b62ca5ae95553be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb536155322d4e1d9b62ca5ae95553be" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42743 } } peers { permanent_uuid: "32138a8519284a09b45a3a01d8e008fe" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 35707 } } peers { permanent_uuid: "c0dbed1d5c444bdfb9b6abf847d57eb3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 41173 } }
I20260619 02:55:03.753764 4132 consensus_queue.cc:260] T abffc195a4b24d7db9864ba5f3979d3a P c0dbed1d5c444bdfb9b6abf847d57eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb536155322d4e1d9b62ca5ae95553be" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42743 } } peers { permanent_uuid: "32138a8519284a09b45a3a01d8e008fe" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 35707 } } peers { permanent_uuid: "c0dbed1d5c444bdfb9b6abf847d57eb3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 41173 } }
I20260619 02:55:03.755396 3979 heartbeater.cc:503] Master 127.18.134.62:44503 was elected leader, sending a full tablet report...
I20260619 02:55:03.756078 4132 ts_tablet_manager.cc:1428] T abffc195a4b24d7db9864ba5f3979d3a P c0dbed1d5c444bdfb9b6abf847d57eb3: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260619 02:55:03.757223 4112 heartbeater.cc:503] Master 127.18.134.62:44503 was elected leader, sending a full tablet report...
I20260619 02:55:03.757791 4131 ts_tablet_manager.cc:1428] T abffc195a4b24d7db9864ba5f3979d3a P eb536155322d4e1d9b62ca5ae95553be: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260619 02:55:03.769979 4136 raft_consensus.cc:482] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:55:03.770239 4136 raft_consensus.cc:504] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb536155322d4e1d9b62ca5ae95553be" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42743 } } peers { permanent_uuid: "32138a8519284a09b45a3a01d8e008fe" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 35707 } } peers { permanent_uuid: "c0dbed1d5c444bdfb9b6abf847d57eb3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 41173 } }
I20260619 02:55:03.771901 4136 leader_election.cc:288] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb536155322d4e1d9b62ca5ae95553be (127.18.134.3:42743), c0dbed1d5c444bdfb9b6abf847d57eb3 (127.18.134.2:41173)
I20260619 02:55:03.778424 4128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:03.772520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:56621 (local address 127.18.134.2:41173)
0619 02:55:03.772717 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:03.772723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:03.773297 (+ 574us) server_negotiation.cc:407] Connection header received
0619 02:55:03.773472 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:03.773476 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:03.773532 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:03.773603 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:03.774172 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.774914 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.776821 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.777577 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.777713 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:03.777799 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:03.778063 (+ 264us) server_negotiation.cc:299] Negotiation successful
0619 02:55:03.778188 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 02:55:03.778584 4139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:03.772313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:41173 (local address 127.18.134.1:56621)
0619 02:55:03.773194 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:03.773209 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:03.773337 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:03.773629 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:03.773633 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:03.773655 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:03.774025 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:03.774038 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.775028 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.775033 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:03.776641 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:03.776653 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.777771 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.777778 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:03.777917 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:03.777943 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 02:55:03.778023 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 02:55:03.778150 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":784,"mutex_wait_us":24,"spinlock_wait_cycles":3968,"thread_start_us":403,"threads_started":1}
I20260619 02:55:03.779246 4126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:03.772320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:43189 (local address 127.18.134.3:42743)
0619 02:55:03.772532 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:03.772539 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:03.773348 (+ 809us) server_negotiation.cc:407] Connection header received
0619 02:55:03.773488 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:03.773491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:03.773560 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:03.773640 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:03.774319 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.775160 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.777381 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.778201 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.778380 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:03.778461 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:03.778839 (+ 378us) server_negotiation.cc:299] Negotiation successful
0619 02:55:03.778966 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 02:55:03.779340 4140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:03.772178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:42743 (local address 127.18.134.1:43189)
0619 02:55:03.773194 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:03.773210 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:03.773359 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:03.773665 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:03.773669 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:03.773693 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:03.774170 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:03.774181 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.775302 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.775308 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:03.777219 (+ 1911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:03.777233 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.778433 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.778440 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:03.778656 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:03.778691 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 02:55:03.778788 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 02:55:03.778915 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"mutex_wait_us":3,"spinlock_wait_cycles":11008,"thread_start_us":721,"threads_started":1}
I20260619 02:55:03.779641 3934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abffc195a4b24d7db9864ba5f3979d3a" candidate_uuid: "32138a8519284a09b45a3a01d8e008fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0dbed1d5c444bdfb9b6abf847d57eb3" is_pre_election: true
I20260619 02:55:03.780184 3934 raft_consensus.cc:2454] T abffc195a4b24d7db9864ba5f3979d3a P c0dbed1d5c444bdfb9b6abf847d57eb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32138a8519284a09b45a3a01d8e008fe in term 0.
I20260619 02:55:03.780392 4067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abffc195a4b24d7db9864ba5f3979d3a" candidate_uuid: "32138a8519284a09b45a3a01d8e008fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb536155322d4e1d9b62ca5ae95553be" is_pre_election: true
I20260619 02:55:03.780810 3736 leader_election.cc:302] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32138a8519284a09b45a3a01d8e008fe, c0dbed1d5c444bdfb9b6abf847d57eb3; no voters:
I20260619 02:55:03.780844 4067 raft_consensus.cc:2454] T abffc195a4b24d7db9864ba5f3979d3a P eb536155322d4e1d9b62ca5ae95553be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32138a8519284a09b45a3a01d8e008fe in term 0.
I20260619 02:55:03.781162 4136 raft_consensus.cc:2781] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:55:03.781296 4136 raft_consensus.cc:482] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:55:03.781397 4136 raft_consensus.cc:3037] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:03.783679 4136 raft_consensus.cc:504] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb536155322d4e1d9b62ca5ae95553be" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42743 } } peers { permanent_uuid: "32138a8519284a09b45a3a01d8e008fe" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 35707 } } peers { permanent_uuid: "c0dbed1d5c444bdfb9b6abf847d57eb3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 41173 } }
I20260619 02:55:03.784324 4136 leader_election.cc:288] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe [CANDIDATE]: Term 1 election: Requested vote from peers eb536155322d4e1d9b62ca5ae95553be (127.18.134.3:42743), c0dbed1d5c444bdfb9b6abf847d57eb3 (127.18.134.2:41173)
I20260619 02:55:03.784859 4067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abffc195a4b24d7db9864ba5f3979d3a" candidate_uuid: "32138a8519284a09b45a3a01d8e008fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb536155322d4e1d9b62ca5ae95553be"
I20260619 02:55:03.785048 4067 raft_consensus.cc:3037] T abffc195a4b24d7db9864ba5f3979d3a P eb536155322d4e1d9b62ca5ae95553be [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:03.785269 3934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abffc195a4b24d7db9864ba5f3979d3a" candidate_uuid: "32138a8519284a09b45a3a01d8e008fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0dbed1d5c444bdfb9b6abf847d57eb3"
I20260619 02:55:03.785496 3934 raft_consensus.cc:3037] T abffc195a4b24d7db9864ba5f3979d3a P c0dbed1d5c444bdfb9b6abf847d57eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:03.787397 4067 raft_consensus.cc:2454] T abffc195a4b24d7db9864ba5f3979d3a P eb536155322d4e1d9b62ca5ae95553be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32138a8519284a09b45a3a01d8e008fe in term 1.
I20260619 02:55:03.787771 3934 raft_consensus.cc:2454] T abffc195a4b24d7db9864ba5f3979d3a P c0dbed1d5c444bdfb9b6abf847d57eb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32138a8519284a09b45a3a01d8e008fe in term 1.
I20260619 02:55:03.787828 3733 leader_election.cc:302] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32138a8519284a09b45a3a01d8e008fe, eb536155322d4e1d9b62ca5ae95553be; no voters:
I20260619 02:55:03.788184 4136 raft_consensus.cc:2781] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:03.788628 4136 raft_consensus.cc:686] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe [term 1 LEADER]: Becoming Leader. State: Replica: 32138a8519284a09b45a3a01d8e008fe, State: Running, Role: LEADER
I20260619 02:55:03.789161 4136 consensus_queue.cc:237] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb536155322d4e1d9b62ca5ae95553be" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42743 } } peers { permanent_uuid: "32138a8519284a09b45a3a01d8e008fe" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 35707 } } peers { permanent_uuid: "c0dbed1d5c444bdfb9b6abf847d57eb3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 41173 } }
I20260619 02:55:03.793924 3655 catalog_manager.cc:5495] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 32138a8519284a09b45a3a01d8e008fe (127.18.134.1). New cstate: current_term: 1 leader_uuid: "32138a8519284a09b45a3a01d8e008fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb536155322d4e1d9b62ca5ae95553be" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32138a8519284a09b45a3a01d8e008fe" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 35707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0dbed1d5c444bdfb9b6abf847d57eb3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260619 02:55:03.816632 4127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:03.811295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44702 (local address 127.18.134.1:35707)
0619 02:55:03.811483 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:03.811489 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:03.811509 (+ 20us) server_negotiation.cc:407] Connection header received
0619 02:55:03.811642 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:03.811646 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:03.811740 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:03.811877 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:55:03.812247 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.812992 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.814434 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.814890 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.815001 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:03.815508 (+ 507us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:55:03.816032 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:55:03.816166 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:03.816267 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 02:55:03.816352 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 02:55:03.824460 3934 raft_consensus.cc:1261] T abffc195a4b24d7db9864ba5f3979d3a P c0dbed1d5c444bdfb9b6abf847d57eb3 [term 1 FOLLOWER]: Refusing update from remote peer 32138a8519284a09b45a3a01d8e008fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:55:03.824460 4067 raft_consensus.cc:1261] T abffc195a4b24d7db9864ba5f3979d3a P eb536155322d4e1d9b62ca5ae95553be [term 1 FOLLOWER]: Refusing update from remote peer 32138a8519284a09b45a3a01d8e008fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:55:03.825414 4141 consensus_queue.cc:1035] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb536155322d4e1d9b62ca5ae95553be" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 42743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:55:03.825744 4136 consensus_queue.cc:1035] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0dbed1d5c444bdfb9b6abf847d57eb3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:55:03.836623 4145 mvcc.cc:205] Tried to move back new op lower bound from 7298407234855628800 to 7298407234724229120. Current Snapshot: MvccSnapshot[applied={T|T < 7298407234855628800}]
I20260619 02:55:03.838531 4146 mvcc.cc:205] Tried to move back new op lower bound from 7298407234855628800 to 7298407234724229120. Current Snapshot: MvccSnapshot[applied={T|T < 7298407234855628800}]
I20260619 02:55:03.847817 3655 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37044:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:55:03.848093 3655 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37044:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:55:03.852563 3655 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c3ff00b9099c4af7ac3d9621cefd4d4e: Sending DeleteTablet for 3 replicas of tablet abffc195a4b24d7db9864ba5f3979d3a
I20260619 02:55:03.854414 4047 tablet_service.cc:1514] Processing DeleteTablet for tablet abffc195a4b24d7db9864ba5f3979d3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:55:03 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:38422
I20260619 02:55:03.855713 3914 tablet_service.cc:1514] Processing DeleteTablet for tablet abffc195a4b24d7db9864ba5f3979d3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:55:03 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:58366
I20260619 02:55:03.857729 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 3717
I20260619 02:55:03.857690 3780 tablet_service.cc:1514] Processing DeleteTablet for tablet abffc195a4b24d7db9864ba5f3979d3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:55:03 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:44688
I20260619 02:55:03.865154 4154 tablet_replica.cc:330] T abffc195a4b24d7db9864ba5f3979d3a P eb536155322d4e1d9b62ca5ae95553be: stopping tablet replica
I20260619 02:55:03.872447 4156 tablet_replica.cc:330] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe: stopping tablet replica
I20260619 02:55:03.872596 4155 tablet_replica.cc:330] T abffc195a4b24d7db9864ba5f3979d3a P c0dbed1d5c444bdfb9b6abf847d57eb3: stopping tablet replica
I20260619 02:55:03.873016 4155 raft_consensus.cc:2229] T abffc195a4b24d7db9864ba5f3979d3a P c0dbed1d5c444bdfb9b6abf847d57eb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:55:03.873355 4156 raft_consensus.cc:2229] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:55:03.873696 4155 raft_consensus.cc:2258] T abffc195a4b24d7db9864ba5f3979d3a P c0dbed1d5c444bdfb9b6abf847d57eb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:55:03.873950 4156 raft_consensus.cc:2258] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:55:03.875900 4154 raft_consensus.cc:2229] T abffc195a4b24d7db9864ba5f3979d3a P eb536155322d4e1d9b62ca5ae95553be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:55:03.876832 4154 raft_consensus.cc:2258] T abffc195a4b24d7db9864ba5f3979d3a P eb536155322d4e1d9b62ca5ae95553be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:55:03.878682 4156 ts_tablet_manager.cc:1905] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:55:03.878904 4155 ts_tablet_manager.cc:1905] T abffc195a4b24d7db9864ba5f3979d3a P c0dbed1d5c444bdfb9b6abf847d57eb3: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 19 02:55:03 dist-test-slave-9mr0 krb5kdc[3611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.1@KRBTEST.COM
I20260619 02:55:03.884295 4156 ts_tablet_manager.cc:1918] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:55:03.884485 4156 log.cc:1196] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-0/wal/wals/abffc195a4b24d7db9864ba5f3979d3a
I20260619 02:55:03.884991 4156 ts_tablet_manager.cc:1939] T abffc195a4b24d7db9864ba5f3979d3a P 32138a8519284a09b45a3a01d8e008fe: Deleting consensus metadata
I20260619 02:55:03.884975 4155 ts_tablet_manager.cc:1918] T abffc195a4b24d7db9864ba5f3979d3a P c0dbed1d5c444bdfb9b6abf847d57eb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:55:03.885119 4155 log.cc:1196] T abffc195a4b24d7db9864ba5f3979d3a P c0dbed1d5c444bdfb9b6abf847d57eb3: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-1/wal/wals/abffc195a4b24d7db9864ba5f3979d3a
I20260619 02:55:03.885596 4155 ts_tablet_manager.cc:1939] T abffc195a4b24d7db9864ba5f3979d3a P c0dbed1d5c444bdfb9b6abf847d57eb3: Deleting consensus metadata
I20260619 02:55:03.885753 4154 ts_tablet_manager.cc:1905] T abffc195a4b24d7db9864ba5f3979d3a P eb536155322d4e1d9b62ca5ae95553be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:55:03.886838 3643 catalog_manager.cc:4841] TS 32138a8519284a09b45a3a01d8e008fe (127.18.134.1:35707): tablet abffc195a4b24d7db9864ba5f3979d3a (table test-table [id=a2e364501b734d3080a1de89df4ddac3]) successfully deleted
I20260619 02:55:03.887152 3643 catalog_manager.cc:4841] TS c0dbed1d5c444bdfb9b6abf847d57eb3 (127.18.134.2:41173): tablet abffc195a4b24d7db9864ba5f3979d3a (table test-table [id=a2e364501b734d3080a1de89df4ddac3]) successfully deleted
I20260619 02:55:03.891424 4127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:03.859134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44718 (local address 127.18.134.1:35707)
0619 02:55:03.859314 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:03.859321 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:03.869731 (+ 10410us) server_negotiation.cc:407] Connection header received
0619 02:55:03.869999 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:03.870005 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:03.870096 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:03.870186 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:03.874276 (+ 4090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.875446 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.877501 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:03.878155 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:03.878287 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:03.883293 (+ 5006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:03.883306 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:03.883322 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:03.883370 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:03.888107 (+ 4737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:03.889011 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:03.889020 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:03.889023 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:03.889201 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:03.889651 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:03.889656 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:03.889660 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:03.890113 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:03.890412 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:03.890770 (+ 358us) server_negotiation.cc:299] Negotiation successful
0619 02:55:03.891011 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 02:55:03.892140 3840 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:03.892482 4154 ts_tablet_manager.cc:1918] T abffc195a4b24d7db9864ba5f3979d3a P eb536155322d4e1d9b62ca5ae95553be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:55:03.892675 4154 log.cc:1196] T abffc195a4b24d7db9864ba5f3979d3a P eb536155322d4e1d9b62ca5ae95553be: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781837410515220-18968-0/minicluster-data/ts-2/wal/wals/abffc195a4b24d7db9864ba5f3979d3a
I20260619 02:55:03.893178 4154 ts_tablet_manager.cc:1939] T abffc195a4b24d7db9864ba5f3979d3a P eb536155322d4e1d9b62ca5ae95553be: Deleting consensus metadata
I20260619 02:55:03.895028 3640 catalog_manager.cc:4841] TS eb536155322d4e1d9b62ca5ae95553be (127.18.134.3:42743): tablet abffc195a4b24d7db9864ba5f3979d3a (table test-table [id=a2e364501b734d3080a1de89df4ddac3]) successfully deleted
I20260619 02:55:03.986769 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 3717
I20260619 02:55:04.001114 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 3851
Jun 19 02:55:04 dist-test-slave-9mr0 krb5kdc[3611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.2@KRBTEST.COM
I20260619 02:55:04.016582 4128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:04.001954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58374 (local address 127.18.134.2:41173)
0619 02:55:04.002211 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:04.002219 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:04.002242 (+ 23us) server_negotiation.cc:407] Connection header received
0619 02:55:04.002370 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:04.002374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:04.002445 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:04.002537 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:04.004057 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:04.004906 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:04.006268 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:04.006754 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:04.006896 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:04.011095 (+ 4199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:04.011111 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:04.011127 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:04.011174 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:04.013827 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:04.014494 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:04.014501 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:04.014504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:04.014844 (+ 340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:04.015181 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:04.015185 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:04.015187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:04.015580 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:04.015807 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:04.016126 (+ 319us) server_negotiation.cc:299] Negotiation successful
0619 02:55:04.016328 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260619 02:55:04.017248 3974 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:04.105895 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 3851
I20260619 02:55:04.119340 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 3984
Jun 19 02:55:04 dist-test-slave-9mr0 krb5kdc[3611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.3@KRBTEST.COM
I20260619 02:55:04.133812 4126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:04.120027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38428 (local address 127.18.134.3:42743)
0619 02:55:04.120203 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:04.120208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:04.120228 (+ 20us) server_negotiation.cc:407] Connection header received
0619 02:55:04.120383 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:04.120388 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:04.120453 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:04.120539 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:04.121818 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:04.122530 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:04.123783 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:04.124168 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:04.124276 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:04.128450 (+ 4174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:04.128466 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:04.128481 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:04.128522 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:04.131219 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:04.131844 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:04.131852 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:04.131854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:04.131988 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:04.132327 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:04.132332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:04.132335 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:04.132688 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:04.132962 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:04.133270 (+ 308us) server_negotiation.cc:299] Negotiation successful
0619 02:55:04.133489 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 02:55:04.134485 4107 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:04.223505 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 3984
I20260619 02:55:04.236614 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 3624
I20260619 02:55:04.237844 3686 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:04.345080 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 3624
2026-06-19T02:55:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (3983 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-19T02:55:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:55:04Z Disabled control of system clock
I20260619 02:55:04.413776 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:36779
--webserver_interface=127.18.134.62
--webserver_port=0
--builtin_ntp_servers=127.18.134.20:45513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:36779
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:55:04.732187 4165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:04.732482 4165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:04.732563 4165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:04.739828 4165 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:04.739935 4165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:04.739971 4165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:04.740031 4165 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:04.740074 4165 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:04.749230 4165 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:45513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:36779
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:36779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:04.750465 4165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:04.752197 4165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:04.763949 4171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:04.763989 4173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:04.763911 4170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:04.764581 4165 server_base.cc:1034] running on GCE node
I20260619 02:55:04.765395 4165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:04.766841 4165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:04.768038 4165 hybrid_clock.cc:648] HybridClock initialized: now 1781837704767986 us; error 51 us; skew 500 ppm
I20260619 02:55:04.768371 4165 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:55:04.770495 4165 webserver.cc:460] Webserver started at http://127.18.134.62:36267/ using document root <none> and password file <none>
I20260619 02:55:04.771185 4165 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:04.771287 4165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:04.771570 4165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:04.774369 4165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "8dd129ea568a4ac3a085de337681fba4"
format_stamp: "Formatted at 2026-06-19 02:55:04 on dist-test-slave-9mr0"
server_key: "352265d3d6e71b306f93e9bd443e556f"
server_key_iv: "ac593f38438fa5d3df3d49323366bccb"
server_key_version: "encryptionkey@0"
I20260619 02:55:04.775174 4165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "8dd129ea568a4ac3a085de337681fba4"
format_stamp: "Formatted at 2026-06-19 02:55:04 on dist-test-slave-9mr0"
server_key: "352265d3d6e71b306f93e9bd443e556f"
server_key_iv: "ac593f38438fa5d3df3d49323366bccb"
server_key_version: "encryptionkey@0"
I20260619 02:55:04.780051 4165 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260619 02:55:04.783304 4179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:04.784566 4165 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260619 02:55:04.784740 4165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "8dd129ea568a4ac3a085de337681fba4"
format_stamp: "Formatted at 2026-06-19 02:55:04 on dist-test-slave-9mr0"
server_key: "352265d3d6e71b306f93e9bd443e556f"
server_key_iv: "ac593f38438fa5d3df3d49323366bccb"
server_key_version: "encryptionkey@0"
I20260619 02:55:04.784932 4165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:04.797056 4165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:04.797827 4165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:04.798069 4165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:04.814532 4165 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:36779
I20260619 02:55:04.814538 4230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:36779 every 8 connection(s)
I20260619 02:55:04.815886 4165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:55:04.819666 4231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:04.825876 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 4165
I20260619 02:55:04.826077 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:55:04.826671 18968 external_mini_cluster.cc:1428] Setting key 1f084ff9fccd311a45b9c3976e147f45
I20260619 02:55:04.830881 4231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dd129ea568a4ac3a085de337681fba4: Bootstrap starting.
I20260619 02:55:04.834280 4231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8dd129ea568a4ac3a085de337681fba4: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:04.835595 4231 log.cc:824] T 00000000000000000000000000000000 P 8dd129ea568a4ac3a085de337681fba4: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:04.839113 4231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dd129ea568a4ac3a085de337681fba4: No bootstrap required, opened a new log
I20260619 02:55:04.840966 4234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:04.828656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53924 (local address 127.18.134.62:36779)
0619 02:55:04.829622 (+ 966us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:04.829642 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:04.829684 (+ 42us) server_negotiation.cc:407] Connection header received
0619 02:55:04.831438 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:04.831461 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:04.831920 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:04.832280 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:04.832900 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:04.834190 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:04.836194 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:04.837268 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:04.837400 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:04.838678 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:04.838688 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:04.838708 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:04.839292 (+ 584us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:04.839372 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:04.839394 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:04.839487 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:04.839722 (+ 235us) server_negotiation.cc:299] Negotiation successful
0619 02:55:04.840023 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":341,"threads_started":1}
I20260619 02:55:04.846583 4231 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8dd129ea568a4ac3a085de337681fba4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dd129ea568a4ac3a085de337681fba4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 36779 } }
I20260619 02:55:04.846993 4231 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8dd129ea568a4ac3a085de337681fba4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:04.847090 4231 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8dd129ea568a4ac3a085de337681fba4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dd129ea568a4ac3a085de337681fba4, State: Initialized, Role: FOLLOWER
I20260619 02:55:04.847657 4231 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8dd129ea568a4ac3a085de337681fba4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dd129ea568a4ac3a085de337681fba4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 36779 } }
I20260619 02:55:04.847889 4231 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8dd129ea568a4ac3a085de337681fba4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:55:04.848014 4231 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8dd129ea568a4ac3a085de337681fba4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:55:04.848173 4231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8dd129ea568a4ac3a085de337681fba4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:04.850436 4231 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8dd129ea568a4ac3a085de337681fba4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dd129ea568a4ac3a085de337681fba4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 36779 } }
I20260619 02:55:04.850977 4231 leader_election.cc:302] T 00000000000000000000000000000000 P 8dd129ea568a4ac3a085de337681fba4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8dd129ea568a4ac3a085de337681fba4; no voters:
I20260619 02:55:04.851552 4231 leader_election.cc:288] T 00000000000000000000000000000000 P 8dd129ea568a4ac3a085de337681fba4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:55:04.851712 4236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8dd129ea568a4ac3a085de337681fba4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:04.852558 4236 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8dd129ea568a4ac3a085de337681fba4 [term 1 LEADER]: Becoming Leader. State: Replica: 8dd129ea568a4ac3a085de337681fba4, State: Running, Role: LEADER
I20260619 02:55:04.853046 4236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8dd129ea568a4ac3a085de337681fba4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dd129ea568a4ac3a085de337681fba4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 36779 } }
I20260619 02:55:04.853603 4231 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8dd129ea568a4ac3a085de337681fba4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:55:04.855891 4237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dd129ea568a4ac3a085de337681fba4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8dd129ea568a4ac3a085de337681fba4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dd129ea568a4ac3a085de337681fba4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 36779 } } }
I20260619 02:55:04.856071 4238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dd129ea568a4ac3a085de337681fba4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8dd129ea568a4ac3a085de337681fba4. Latest consensus state: current_term: 1 leader_uuid: "8dd129ea568a4ac3a085de337681fba4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dd129ea568a4ac3a085de337681fba4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 36779 } } }
I20260619 02:55:04.856374 4237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dd129ea568a4ac3a085de337681fba4 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:04.856401 4238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dd129ea568a4ac3a085de337681fba4 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:04.859226 4245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:55:04.863811 4245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:55:04.880972 4245 catalog_manager.cc:1347] Generated new cluster ID: b9e634f23609486ca345a67854c03a10
I20260619 02:55:04.881108 4245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:55:04.901687 4245 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:55:04.903496 4245 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:55:04.914698 4245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8dd129ea568a4ac3a085de337681fba4: Generated new TSK 0
I20260619 02:55:04.915755 4245 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:55:04.928166 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:0
--local_ip_for_outbound_sockets=127.18.134.1
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:36779
--builtin_ntp_servers=127.18.134.20:45513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:55:05.266099 4255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:05.266392 4255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:05.266511 4255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:05.273738 4255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:05.273851 4255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:05.273986 4255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:55:05.283160 4255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:45513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:36779
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:05.284435 4255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:05.286139 4255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:05.298810 4260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:05.298834 4261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:05.298759 4263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:05.299575 4255 server_base.cc:1034] running on GCE node
I20260619 02:55:05.300137 4255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:05.301056 4255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:05.302258 4255 hybrid_clock.cc:648] HybridClock initialized: now 1781837705302211 us; error 39 us; skew 500 ppm
I20260619 02:55:05.302551 4255 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:55:05.311405 4255 webserver.cc:460] Webserver started at http://127.18.134.1:35213/ using document root <none> and password file <none>
I20260619 02:55:05.312157 4255 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:05.312275 4255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:05.312615 4255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:05.315683 4255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-0/data/instance:
uuid: "dc1404916c4740778e0ba1a8b94d803e"
format_stamp: "Formatted at 2026-06-19 02:55:05 on dist-test-slave-9mr0"
server_key: "93b10ad6d6b8984350a0ab50a972f548"
server_key_iv: "31803fc3c1d7a4625f1d0f9fcf647a57"
server_key_version: "encryptionkey@0"
I20260619 02:55:05.316488 4255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance:
uuid: "dc1404916c4740778e0ba1a8b94d803e"
format_stamp: "Formatted at 2026-06-19 02:55:05 on dist-test-slave-9mr0"
server_key: "93b10ad6d6b8984350a0ab50a972f548"
server_key_iv: "31803fc3c1d7a4625f1d0f9fcf647a57"
server_key_version: "encryptionkey@0"
I20260619 02:55:05.321935 4255 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260619 02:55:05.325476 4269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:05.326896 4255 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:55:05.327114 4255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "dc1404916c4740778e0ba1a8b94d803e"
format_stamp: "Formatted at 2026-06-19 02:55:05 on dist-test-slave-9mr0"
server_key: "93b10ad6d6b8984350a0ab50a972f548"
server_key_iv: "31803fc3c1d7a4625f1d0f9fcf647a57"
server_key_version: "encryptionkey@0"
I20260619 02:55:05.327337 4255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:05.346477 4255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:05.347419 4255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:05.347764 4255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:05.348781 4255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:55:05.350449 4255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:55:05.350566 4255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:05.350698 4255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:55:05.350769 4255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:05.380626 4255 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:46777
I20260619 02:55:05.380676 4381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:46777 every 8 connection(s)
I20260619 02:55:05.381915 4255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:55:05.383026 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 4255
I20260619 02:55:05.383203 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance
I20260619 02:55:05.383759 18968 external_mini_cluster.cc:1428] Setting key b99b20fcfc92b2697a8a817a8358df62
I20260619 02:55:05.386871 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:0
--local_ip_for_outbound_sockets=127.18.134.2
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:36779
--builtin_ntp_servers=127.18.134.20:45513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 02:55:05.394657 4384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:05.384624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:33215 (local address 127.18.134.62:36779)
0619 02:55:05.385058 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:05.385062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:05.386486 (+ 1424us) server_negotiation.cc:407] Connection header received
0619 02:55:05.388285 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:05.388290 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:05.388372 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:05.388464 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:05.389547 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:05.390274 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:05.391813 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:05.392210 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:05.392326 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:05.393754 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:05.393760 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:05.393763 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:05.393781 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:05.393825 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:05.393846 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:05.393922 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:05.394150 (+ 228us) server_negotiation.cc:299] Negotiation successful
0619 02:55:05.394290 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":227,"threads_started":1}
I20260619 02:55:05.395581 4385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:05.385462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:36779 (local address 127.18.134.1:33215)
0619 02:55:05.386346 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:05.386383 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:05.388008 (+ 1625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:05.388719 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:05.388734 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:05.388822 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:05.389394 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:05.389413 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:05.390420 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:05.390424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:05.391674 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:05.391686 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:05.392427 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:05.392434 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:05.392677 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:05.393401 (+ 724us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:55:05.393405 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:05.393458 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:55:05.393564 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:05.393934 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:05.393941 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:05.393947 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 02:55:05.394106 (+ 159us) client_negotiation.cc:240] Negotiation successful
0619 02:55:05.394492 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":273,"threads_started":1}
I20260619 02:55:05.397079 4382 heartbeater.cc:347] Connected to a master server at 127.18.134.62:36779
I20260619 02:55:05.397365 4382 heartbeater.cc:464] Registering TS with master...
I20260619 02:55:05.398105 4382 heartbeater.cc:511] Master 127.18.134.62:36779 requested a full tablet report, sending...
I20260619 02:55:05.400264 4196 ts_manager.cc:194] Registered new tserver with Master: dc1404916c4740778e0ba1a8b94d803e (127.18.134.1:46777)
I20260619 02:55:05.402268 4196 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.134.1:33215
W20260619 02:55:05.701046 4386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:05.701337 4386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:05.701432 4386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:05.708666 4386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:05.708777 4386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:05.708923 4386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:55:05.718003 4386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:45513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:36779
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:05.719341 4386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:05.721133 4386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:05.733596 4394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:05.733687 4391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:05.733680 4392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:05.734309 4386 server_base.cc:1034] running on GCE node
I20260619 02:55:05.734922 4386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:05.735828 4386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:05.737023 4386 hybrid_clock.cc:648] HybridClock initialized: now 1781837705736984 us; error 32 us; skew 500 ppm
I20260619 02:55:05.737306 4386 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:55:05.739495 4386 webserver.cc:460] Webserver started at http://127.18.134.2:42589/ using document root <none> and password file <none>
I20260619 02:55:05.740131 4386 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:05.740216 4386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:05.740494 4386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:05.743319 4386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-1/data/instance:
uuid: "54f8cdc8b1b64145a4e88bdea2862fc3"
format_stamp: "Formatted at 2026-06-19 02:55:05 on dist-test-slave-9mr0"
server_key: "35ea8c8d5c3fa28a9c4c3fa08d68d54b"
server_key_iv: "a4ff62870658ed5df612ad62f4b33a29"
server_key_version: "encryptionkey@0"
I20260619 02:55:05.744055 4386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance:
uuid: "54f8cdc8b1b64145a4e88bdea2862fc3"
format_stamp: "Formatted at 2026-06-19 02:55:05 on dist-test-slave-9mr0"
server_key: "35ea8c8d5c3fa28a9c4c3fa08d68d54b"
server_key_iv: "a4ff62870658ed5df612ad62f4b33a29"
server_key_version: "encryptionkey@0"
I20260619 02:55:05.749017 4386 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260619 02:55:05.752422 4400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:05.753769 4386 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 02:55:05.753911 4386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "54f8cdc8b1b64145a4e88bdea2862fc3"
format_stamp: "Formatted at 2026-06-19 02:55:05 on dist-test-slave-9mr0"
server_key: "35ea8c8d5c3fa28a9c4c3fa08d68d54b"
server_key_iv: "a4ff62870658ed5df612ad62f4b33a29"
server_key_version: "encryptionkey@0"
I20260619 02:55:05.754086 4386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:05.765198 4386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:05.765937 4386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:05.766176 4386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:05.767144 4386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:55:05.768489 4386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:55:05.768572 4386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:05.768656 4386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:55:05.768707 4386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:05.798830 4386 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:40781
I20260619 02:55:05.798892 4512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:40781 every 8 connection(s)
I20260619 02:55:05.800146 4386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:55:05.800565 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 4386
I20260619 02:55:05.800724 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance
I20260619 02:55:05.801283 18968 external_mini_cluster.cc:1428] Setting key 1fc0a6a7761588a0b666158aa742ff61
I20260619 02:55:05.804111 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:0
--local_ip_for_outbound_sockets=127.18.134.3
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:36779
--builtin_ntp_servers=127.18.134.20:45513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 02:55:05.812942 4384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:05.802603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:47317 (local address 127.18.134.62:36779)
0619 02:55:05.802791 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:05.802796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:05.804422 (+ 1626us) server_negotiation.cc:407] Connection header received
0619 02:55:05.806125 (+ 1703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:05.806130 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:05.806206 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:05.806316 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:05.807407 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:05.808394 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:05.809989 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:05.810595 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:05.810755 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:05.812235 (+ 1480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:05.812240 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:05.812243 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:05.812292 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:05.812322 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:05.812333 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:05.812401 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:05.812625 (+ 224us) server_negotiation.cc:299] Negotiation successful
0619 02:55:05.812710 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 02:55:05.814319 4515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:05.803275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:36779 (local address 127.18.134.2:47317)
0619 02:55:05.804279 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:05.804321 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:05.805875 (+ 1554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:05.806523 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:05.806533 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:05.806683 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:05.807252 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:05.807271 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:05.808562 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:05.808567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:05.809844 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:05.809859 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:05.810851 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:05.810859 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:05.811150 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:05.811871 (+ 721us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:55:05.811874 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:05.811938 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:55:05.812054 (+ 116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:05.812416 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:05.812426 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:05.812433 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 02:55:05.812628 (+ 195us) client_negotiation.cc:240] Negotiation successful
0619 02:55:05.813015 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":289,"threads_started":1}
I20260619 02:55:05.819659 4513 heartbeater.cc:347] Connected to a master server at 127.18.134.62:36779
I20260619 02:55:05.820034 4513 heartbeater.cc:464] Registering TS with master...
I20260619 02:55:05.820974 4513 heartbeater.cc:511] Master 127.18.134.62:36779 requested a full tablet report, sending...
I20260619 02:55:05.822782 4196 ts_manager.cc:194] Registered new tserver with Master: 54f8cdc8b1b64145a4e88bdea2862fc3 (127.18.134.2:40781)
I20260619 02:55:05.823908 4196 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.134.2:47317
W20260619 02:55:06.124687 4516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:06.124984 4516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:06.125108 4516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:06.132550 4516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:06.132664 4516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:06.132819 4516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:55:06.142122 4516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:45513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:36779
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:06.143416 4516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:06.145148 4516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:06.157779 4521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:06.157879 4524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:06.157903 4522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:06.158989 4516 server_base.cc:1034] running on GCE node
I20260619 02:55:06.159572 4516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:06.160517 4516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:06.161723 4516 hybrid_clock.cc:648] HybridClock initialized: now 1781837706161651 us; error 63 us; skew 500 ppm
I20260619 02:55:06.162035 4516 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:55:06.164294 4516 webserver.cc:460] Webserver started at http://127.18.134.3:42931/ using document root <none> and password file <none>
I20260619 02:55:06.164968 4516 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:06.165067 4516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:06.165369 4516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:06.168326 4516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-2/data/instance:
uuid: "31bac92b1cd7471b8c7ff8bfa3c50df8"
format_stamp: "Formatted at 2026-06-19 02:55:06 on dist-test-slave-9mr0"
server_key: "8b39c376425e89aa23350b0e3ce2c0e9"
server_key_iv: "73dd2c1e971b438859245812bfa02bd6"
server_key_version: "encryptionkey@0"
I20260619 02:55:06.169100 4516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance:
uuid: "31bac92b1cd7471b8c7ff8bfa3c50df8"
format_stamp: "Formatted at 2026-06-19 02:55:06 on dist-test-slave-9mr0"
server_key: "8b39c376425e89aa23350b0e3ce2c0e9"
server_key_iv: "73dd2c1e971b438859245812bfa02bd6"
server_key_version: "encryptionkey@0"
I20260619 02:55:06.174083 4516 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 02:55:06.177598 4530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:06.178969 4516 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 02:55:06.179119 4516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "31bac92b1cd7471b8c7ff8bfa3c50df8"
format_stamp: "Formatted at 2026-06-19 02:55:06 on dist-test-slave-9mr0"
server_key: "8b39c376425e89aa23350b0e3ce2c0e9"
server_key_iv: "73dd2c1e971b438859245812bfa02bd6"
server_key_version: "encryptionkey@0"
I20260619 02:55:06.179312 4516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:06.191648 4516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:06.192504 4516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:06.192796 4516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:06.193751 4516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:55:06.195257 4516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:55:06.195353 4516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:06.195464 4516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:55:06.195533 4516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:06.224974 4516 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:34559
I20260619 02:55:06.225023 4642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:34559 every 8 connection(s)
I20260619 02:55:06.226217 4516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:55:06.228587 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 4516
I20260619 02:55:06.228766 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance
I20260619 02:55:06.229220 18968 external_mini_cluster.cc:1428] Setting key a113e95c6874a380091f212416c8eac3
I20260619 02:55:06.237989 4384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:06.228654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:60189 (local address 127.18.134.62:36779)
0619 02:55:06.228862 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:06.228868 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:06.230373 (+ 1505us) server_negotiation.cc:407] Connection header received
0619 02:55:06.232347 (+ 1974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:06.232351 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:06.232417 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:06.232494 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:06.233508 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.234220 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.235751 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.236151 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.236241 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:06.237394 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:06.237398 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:06.237402 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:06.237418 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:06.237450 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:06.237461 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:06.237526 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:06.237676 (+ 150us) server_negotiation.cc:299] Negotiation successful
0619 02:55:06.237754 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260619 02:55:06.238979 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:06.229158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:36779 (local address 127.18.134.3:60189)
0619 02:55:06.230241 (+ 1083us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:06.230285 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:06.232088 (+ 1803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:06.232677 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:06.232685 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:06.232775 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:06.233363 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:06.233382 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.234372 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.234376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:06.235642 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:06.235653 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.236314 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.236319 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:06.236510 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:06.237115 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:55:06.237118 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:06.237160 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:55:06.237232 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:06.237530 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:06.237535 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:06.237539 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 02:55:06.237667 (+ 128us) client_negotiation.cc:240] Negotiation successful
0619 02:55:06.237935 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":310,"threads_started":1}
I20260619 02:55:06.240403 4643 heartbeater.cc:347] Connected to a master server at 127.18.134.62:36779
I20260619 02:55:06.240681 4643 heartbeater.cc:464] Registering TS with master...
I20260619 02:55:06.241365 4643 heartbeater.cc:511] Master 127.18.134.62:36779 requested a full tablet report, sending...
I20260619 02:55:06.242758 4196 ts_manager.cc:194] Registered new tserver with Master: 31bac92b1cd7471b8c7ff8bfa3c50df8 (127.18.134.3:34559)
I20260619 02:55:06.243683 4196 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.134.3:60189
I20260619 02:55:06.244326 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:55:06.253509 4384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:06.248104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53936 (local address 127.18.134.62:36779)
0619 02:55:06.248326 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:06.248332 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:06.248649 (+ 317us) server_negotiation.cc:407] Connection header received
0619 02:55:06.248758 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:06.248762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:06.248833 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:06.248911 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:06.249343 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.250015 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.251460 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.251897 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.251997 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:06.252913 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:06.252918 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:06.252921 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:06.252936 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:06.252964 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:06.252974 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:06.253040 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:06.253120 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 02:55:06.253199 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 02:55:06.257009 4195 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53936:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:55:06.263671 4195 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:55:06.290663 4657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:06.275057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:46777 (local address 127.0.0.1:49290)
0619 02:55:06.276202 (+ 1145us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:06.276239 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:06.276415 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:06.282841 (+ 6426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:06.282849 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:06.282881 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:06.283307 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:06.283322 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.285423 (+ 2101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.285428 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:06.287705 (+ 2277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:06.287719 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.289649 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.289655 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:06.289861 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:06.289946 (+ 85us) client_negotiation.cc:769] Sending connection context
0619 02:55:06.290042 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 02:55:06.290236 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":904,"mutex_wait_us":3,"thread_start_us":6832,"threads_started":1}
I20260619 02:55:06.291962 4656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:06.275139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49290 (local address 127.18.134.1:46777)
0619 02:55:06.282058 (+ 6919us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:06.282066 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:06.282095 (+ 29us) server_negotiation.cc:407] Connection header received
0619 02:55:06.282210 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:06.282219 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:06.282501 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:06.283533 (+ 1032us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:06.283814 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.285264 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.287926 (+ 2662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.289449 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.291140 (+ 1691us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:06.291398 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:06.291471 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 02:55:06.291633 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6757,"thread_start_us":6637,"threads_started":1}
I20260619 02:55:06.292970 4658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:06.276066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57062 (local address 127.18.134.2:40781)
0619 02:55:06.281735 (+ 5669us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:06.281756 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:06.282287 (+ 531us) server_negotiation.cc:407] Connection header received
0619 02:55:06.283764 (+ 1477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:06.283772 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:06.284022 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:06.284176 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:06.285079 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.287605 (+ 2526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.290138 (+ 2533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.291517 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.291616 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:06.291871 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:06.292469 (+ 598us) server_negotiation.cc:299] Negotiation successful
0619 02:55:06.292627 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5510,"thread_start_us":5398,"threads_started":1}
I20260619 02:55:06.293690 4655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:06.275592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:40781 (local address 127.0.0.1:57062)
0619 02:55:06.282177 (+ 6585us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:06.282194 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:06.283560 (+ 1366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:06.284491 (+ 931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:06.284496 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:06.284520 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:06.284910 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:06.284921 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.287793 (+ 2872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.287798 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:06.289934 (+ 2136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:06.289946 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.292058 (+ 2112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.292065 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:06.292250 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:06.292287 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 02:55:06.292372 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 02:55:06.293422 (+ 1050us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6456,"mutex_wait_us":962,"thread_start_us":589,"threads_started":1}
I20260619 02:55:06.296232 4659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:06.282268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:34559 (local address 127.0.0.1:52178)
0619 02:55:06.288879 (+ 6611us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:06.288900 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:06.289017 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:06.290099 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:06.290104 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:06.290138 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:06.290455 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:06.290464 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.291977 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.291982 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:06.293850 (+ 1868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:06.293864 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.295528 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.295534 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:06.295700 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:06.295732 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 02:55:06.295822 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 02:55:06.295942 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6473,"thread_start_us":6823,"threads_started":1}
I20260619 02:55:06.296377 4660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:06.282676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52178 (local address 127.18.134.3:34559)
0619 02:55:06.289659 (+ 6983us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:06.289664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:06.289690 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:55:06.289764 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:06.289769 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:06.289928 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:06.290092 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:06.290651 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.291836 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.294039 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.295349 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.295466 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:06.295705 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:06.295880 (+ 175us) server_negotiation.cc:299] Negotiation successful
0619 02:55:06.296022 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6836,"thread_start_us":6694,"threads_started":1}
I20260619 02:55:06.295812 4317 tablet_service.cc:1467] Processing CreateTablet for tablet ba7bc65e03fa461eb3ad1a1f23a80c4c (DEFAULT_TABLE table=test-table [id=493094e9e39445baa9384c5c80d46da0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:55:06.297264 4317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba7bc65e03fa461eb3ad1a1f23a80c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:06.297317 4448 tablet_service.cc:1467] Processing CreateTablet for tablet ba7bc65e03fa461eb3ad1a1f23a80c4c (DEFAULT_TABLE table=test-table [id=493094e9e39445baa9384c5c80d46da0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:55:06.298712 4448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba7bc65e03fa461eb3ad1a1f23a80c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:06.299506 4578 tablet_service.cc:1467] Processing CreateTablet for tablet ba7bc65e03fa461eb3ad1a1f23a80c4c (DEFAULT_TABLE table=test-table [id=493094e9e39445baa9384c5c80d46da0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:55:06.300544 4578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba7bc65e03fa461eb3ad1a1f23a80c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:06.311236 4661 tablet_bootstrap.cc:492] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 31bac92b1cd7471b8c7ff8bfa3c50df8: Bootstrap starting.
I20260619 02:55:06.314550 4661 tablet_bootstrap.cc:654] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 31bac92b1cd7471b8c7ff8bfa3c50df8: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:06.315764 4661 log.cc:824] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 31bac92b1cd7471b8c7ff8bfa3c50df8: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:06.318079 4662 tablet_bootstrap.cc:492] T ba7bc65e03fa461eb3ad1a1f23a80c4c P dc1404916c4740778e0ba1a8b94d803e: Bootstrap starting.
I20260619 02:55:06.318666 4661 tablet_bootstrap.cc:492] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 31bac92b1cd7471b8c7ff8bfa3c50df8: No bootstrap required, opened a new log
I20260619 02:55:06.318704 4663 tablet_bootstrap.cc:492] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3: Bootstrap starting.
I20260619 02:55:06.318934 4661 ts_tablet_manager.cc:1397] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 31bac92b1cd7471b8c7ff8bfa3c50df8: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260619 02:55:06.321610 4663 tablet_bootstrap.cc:654] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:06.321620 4662 tablet_bootstrap.cc:654] T ba7bc65e03fa461eb3ad1a1f23a80c4c P dc1404916c4740778e0ba1a8b94d803e: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:06.322803 4662 log.cc:824] T ba7bc65e03fa461eb3ad1a1f23a80c4c P dc1404916c4740778e0ba1a8b94d803e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:06.322803 4663 log.cc:824] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:06.325801 4663 tablet_bootstrap.cc:492] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3: No bootstrap required, opened a new log
I20260619 02:55:06.326035 4663 ts_tablet_manager.cc:1397] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260619 02:55:06.326370 4662 tablet_bootstrap.cc:492] T ba7bc65e03fa461eb3ad1a1f23a80c4c P dc1404916c4740778e0ba1a8b94d803e: No bootstrap required, opened a new log
I20260619 02:55:06.326646 4662 ts_tablet_manager.cc:1397] T ba7bc65e03fa461eb3ad1a1f23a80c4c P dc1404916c4740778e0ba1a8b94d803e: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260619 02:55:06.327178 4661 raft_consensus.cc:348] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 31bac92b1cd7471b8c7ff8bfa3c50df8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc1404916c4740778e0ba1a8b94d803e" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 46777 } } peers { permanent_uuid: "31bac92b1cd7471b8c7ff8bfa3c50df8" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34559 } } peers { permanent_uuid: "54f8cdc8b1b64145a4e88bdea2862fc3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 40781 } }
I20260619 02:55:06.327636 4661 raft_consensus.cc:374] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 31bac92b1cd7471b8c7ff8bfa3c50df8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:06.327744 4661 raft_consensus.cc:729] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 31bac92b1cd7471b8c7ff8bfa3c50df8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31bac92b1cd7471b8c7ff8bfa3c50df8, State: Initialized, Role: FOLLOWER
I20260619 02:55:06.328374 4661 consensus_queue.cc:260] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 31bac92b1cd7471b8c7ff8bfa3c50df8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc1404916c4740778e0ba1a8b94d803e" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 46777 } } peers { permanent_uuid: "31bac92b1cd7471b8c7ff8bfa3c50df8" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34559 } } peers { permanent_uuid: "54f8cdc8b1b64145a4e88bdea2862fc3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 40781 } }
I20260619 02:55:06.333362 4663 raft_consensus.cc:348] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc1404916c4740778e0ba1a8b94d803e" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 46777 } } peers { permanent_uuid: "31bac92b1cd7471b8c7ff8bfa3c50df8" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34559 } } peers { permanent_uuid: "54f8cdc8b1b64145a4e88bdea2862fc3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 40781 } }
I20260619 02:55:06.333688 4663 raft_consensus.cc:374] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:06.333786 4663 raft_consensus.cc:729] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54f8cdc8b1b64145a4e88bdea2862fc3, State: Initialized, Role: FOLLOWER
I20260619 02:55:06.334326 4663 consensus_queue.cc:260] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc1404916c4740778e0ba1a8b94d803e" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 46777 } } peers { permanent_uuid: "31bac92b1cd7471b8c7ff8bfa3c50df8" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34559 } } peers { permanent_uuid: "54f8cdc8b1b64145a4e88bdea2862fc3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 40781 } }
I20260619 02:55:06.335331 4513 heartbeater.cc:503] Master 127.18.134.62:36779 was elected leader, sending a full tablet report...
I20260619 02:55:06.336066 4661 ts_tablet_manager.cc:1428] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 31bac92b1cd7471b8c7ff8bfa3c50df8: Time spent starting tablet: real 0.017s user 0.008s sys 0.008s
I20260619 02:55:06.336230 4663 ts_tablet_manager.cc:1428] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260619 02:55:06.336412 4643 heartbeater.cc:503] Master 127.18.134.62:36779 was elected leader, sending a full tablet report...
I20260619 02:55:06.337201 4662 raft_consensus.cc:348] T ba7bc65e03fa461eb3ad1a1f23a80c4c P dc1404916c4740778e0ba1a8b94d803e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc1404916c4740778e0ba1a8b94d803e" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 46777 } } peers { permanent_uuid: "31bac92b1cd7471b8c7ff8bfa3c50df8" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34559 } } peers { permanent_uuid: "54f8cdc8b1b64145a4e88bdea2862fc3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 40781 } }
I20260619 02:55:06.337612 4662 raft_consensus.cc:374] T ba7bc65e03fa461eb3ad1a1f23a80c4c P dc1404916c4740778e0ba1a8b94d803e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:06.337703 4662 raft_consensus.cc:729] T ba7bc65e03fa461eb3ad1a1f23a80c4c P dc1404916c4740778e0ba1a8b94d803e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc1404916c4740778e0ba1a8b94d803e, State: Initialized, Role: FOLLOWER
I20260619 02:55:06.338290 4662 consensus_queue.cc:260] T ba7bc65e03fa461eb3ad1a1f23a80c4c P dc1404916c4740778e0ba1a8b94d803e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc1404916c4740778e0ba1a8b94d803e" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 46777 } } peers { permanent_uuid: "31bac92b1cd7471b8c7ff8bfa3c50df8" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34559 } } peers { permanent_uuid: "54f8cdc8b1b64145a4e88bdea2862fc3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 40781 } }
I20260619 02:55:06.339783 4382 heartbeater.cc:503] Master 127.18.134.62:36779 was elected leader, sending a full tablet report...
I20260619 02:55:06.340055 4662 ts_tablet_manager.cc:1428] T ba7bc65e03fa461eb3ad1a1f23a80c4c P dc1404916c4740778e0ba1a8b94d803e: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
W20260619 02:55:06.383910 4383 tablet.cc:2369] T ba7bc65e03fa461eb3ad1a1f23a80c4c P dc1404916c4740778e0ba1a8b94d803e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:55:06.466092 4668 raft_consensus.cc:482] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:55:06.466403 4668 raft_consensus.cc:504] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc1404916c4740778e0ba1a8b94d803e" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 46777 } } peers { permanent_uuid: "31bac92b1cd7471b8c7ff8bfa3c50df8" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34559 } } peers { permanent_uuid: "54f8cdc8b1b64145a4e88bdea2862fc3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 40781 } }
I20260619 02:55:06.467983 4668 leader_election.cc:288] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc1404916c4740778e0ba1a8b94d803e (127.18.134.1:46777), 31bac92b1cd7471b8c7ff8bfa3c50df8 (127.18.134.3:34559)
I20260619 02:55:06.474001 4656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:06.468414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:57007 (local address 127.18.134.1:46777)
0619 02:55:06.468588 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:06.468592 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:06.468959 (+ 367us) server_negotiation.cc:407] Connection header received
0619 02:55:06.469131 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:06.469136 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:06.469228 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:06.469326 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:06.469910 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.470581 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.472410 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.473181 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.473316 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:06.473384 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:06.473627 (+ 243us) server_negotiation.cc:299] Negotiation successful
0619 02:55:06.473769 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 02:55:06.474045 4670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:06.468202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:46777 (local address 127.18.134.2:57007)
0619 02:55:06.468827 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:06.468846 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:06.468983 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:06.469430 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:06.469435 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:06.469464 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:06.469774 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:06.469783 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.470743 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.470748 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:06.472230 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:06.472242 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.473352 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.473358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:06.473492 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:06.473515 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 02:55:06.473592 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 02:55:06.473719 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":302,"threads_started":1}
I20260619 02:55:06.474387 4660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:06.469112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:57901 (local address 127.18.134.3:34559)
0619 02:55:06.469327 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:06.469332 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:06.469603 (+ 271us) server_negotiation.cc:407] Connection header received
0619 02:55:06.469769 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:06.469773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:06.469832 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:06.469904 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:06.470421 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.471138 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.472780 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.473544 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.473696 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:06.473762 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:06.474029 (+ 267us) server_negotiation.cc:299] Negotiation successful
0619 02:55:06.474154 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260619 02:55:06.474679 4671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:06.468902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:34559 (local address 127.18.134.2:57901)
0619 02:55:06.469498 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:06.469515 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:06.469636 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:06.469932 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:06.469936 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:06.469958 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:06.470264 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:06.470272 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.471273 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.471278 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:06.472608 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:06.472620 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.473722 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.473728 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:06.473887 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:06.473910 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 02:55:06.473984 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 02:55:06.474393 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":353,"threads_started":1}
I20260619 02:55:06.475440 4337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba7bc65e03fa461eb3ad1a1f23a80c4c" candidate_uuid: "54f8cdc8b1b64145a4e88bdea2862fc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc1404916c4740778e0ba1a8b94d803e" is_pre_election: true
I20260619 02:55:06.475662 4598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba7bc65e03fa461eb3ad1a1f23a80c4c" candidate_uuid: "54f8cdc8b1b64145a4e88bdea2862fc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31bac92b1cd7471b8c7ff8bfa3c50df8" is_pre_election: true
I20260619 02:55:06.475927 4337 raft_consensus.cc:2454] T ba7bc65e03fa461eb3ad1a1f23a80c4c P dc1404916c4740778e0ba1a8b94d803e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54f8cdc8b1b64145a4e88bdea2862fc3 in term 0.
I20260619 02:55:06.476138 4598 raft_consensus.cc:2454] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 31bac92b1cd7471b8c7ff8bfa3c50df8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54f8cdc8b1b64145a4e88bdea2862fc3 in term 0.
I20260619 02:55:06.476586 4401 leader_election.cc:302] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54f8cdc8b1b64145a4e88bdea2862fc3, dc1404916c4740778e0ba1a8b94d803e; no voters:
I20260619 02:55:06.476946 4668 raft_consensus.cc:2781] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:55:06.477064 4668 raft_consensus.cc:482] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:55:06.477164 4668 raft_consensus.cc:3037] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3 [term 0 FOLLOWER]: Advancing to term 1
W20260619 02:55:06.477770 4644 tablet.cc:2369] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 31bac92b1cd7471b8c7ff8bfa3c50df8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:55:06.479691 4668 raft_consensus.cc:504] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc1404916c4740778e0ba1a8b94d803e" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 46777 } } peers { permanent_uuid: "31bac92b1cd7471b8c7ff8bfa3c50df8" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34559 } } peers { permanent_uuid: "54f8cdc8b1b64145a4e88bdea2862fc3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 40781 } }
I20260619 02:55:06.480226 4668 leader_election.cc:288] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3 [CANDIDATE]: Term 1 election: Requested vote from peers dc1404916c4740778e0ba1a8b94d803e (127.18.134.1:46777), 31bac92b1cd7471b8c7ff8bfa3c50df8 (127.18.134.3:34559)
I20260619 02:55:06.480684 4337 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba7bc65e03fa461eb3ad1a1f23a80c4c" candidate_uuid: "54f8cdc8b1b64145a4e88bdea2862fc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc1404916c4740778e0ba1a8b94d803e"
I20260619 02:55:06.480736 4598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba7bc65e03fa461eb3ad1a1f23a80c4c" candidate_uuid: "54f8cdc8b1b64145a4e88bdea2862fc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31bac92b1cd7471b8c7ff8bfa3c50df8"
I20260619 02:55:06.480882 4337 raft_consensus.cc:3037] T ba7bc65e03fa461eb3ad1a1f23a80c4c P dc1404916c4740778e0ba1a8b94d803e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:06.480898 4598 raft_consensus.cc:3037] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 31bac92b1cd7471b8c7ff8bfa3c50df8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:06.483168 4598 raft_consensus.cc:2454] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 31bac92b1cd7471b8c7ff8bfa3c50df8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54f8cdc8b1b64145a4e88bdea2862fc3 in term 1.
I20260619 02:55:06.483172 4337 raft_consensus.cc:2454] T ba7bc65e03fa461eb3ad1a1f23a80c4c P dc1404916c4740778e0ba1a8b94d803e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54f8cdc8b1b64145a4e88bdea2862fc3 in term 1.
I20260619 02:55:06.483487 4401 leader_election.cc:302] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54f8cdc8b1b64145a4e88bdea2862fc3, dc1404916c4740778e0ba1a8b94d803e; no voters:
I20260619 02:55:06.483752 4668 raft_consensus.cc:2781] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:06.484171 4668 raft_consensus.cc:686] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3 [term 1 LEADER]: Becoming Leader. State: Replica: 54f8cdc8b1b64145a4e88bdea2862fc3, State: Running, Role: LEADER
I20260619 02:55:06.484616 4668 consensus_queue.cc:237] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc1404916c4740778e0ba1a8b94d803e" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 46777 } } peers { permanent_uuid: "31bac92b1cd7471b8c7ff8bfa3c50df8" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34559 } } peers { permanent_uuid: "54f8cdc8b1b64145a4e88bdea2862fc3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 40781 } }
I20260619 02:55:06.489599 4196 catalog_manager.cc:5495] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 54f8cdc8b1b64145a4e88bdea2862fc3 (127.18.134.2). New cstate: current_term: 1 leader_uuid: "54f8cdc8b1b64145a4e88bdea2862fc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc1404916c4740778e0ba1a8b94d803e" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 46777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31bac92b1cd7471b8c7ff8bfa3c50df8" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54f8cdc8b1b64145a4e88bdea2862fc3" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 40781 } health_report { overall_health: HEALTHY } } }
I20260619 02:55:06.523681 4658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:06.518134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57076 (local address 127.18.134.2:40781)
0619 02:55:06.518319 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:06.518325 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:06.518381 (+ 56us) server_negotiation.cc:407] Connection header received
0619 02:55:06.518594 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:06.518598 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:06.518725 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:06.518855 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:55:06.519256 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.519965 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.521552 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.521964 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.522072 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:06.522557 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:55:06.523120 (+ 563us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:55:06.523263 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:06.523359 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 02:55:06.523438 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 02:55:06.532012 4337 raft_consensus.cc:1261] T ba7bc65e03fa461eb3ad1a1f23a80c4c P dc1404916c4740778e0ba1a8b94d803e [term 1 FOLLOWER]: Refusing update from remote peer 54f8cdc8b1b64145a4e88bdea2862fc3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:55:06.532022 4598 raft_consensus.cc:1261] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 31bac92b1cd7471b8c7ff8bfa3c50df8 [term 1 FOLLOWER]: Refusing update from remote peer 54f8cdc8b1b64145a4e88bdea2862fc3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:55:06.532912 4672 consensus_queue.cc:1035] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc1404916c4740778e0ba1a8b94d803e" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 46777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:55:06.533336 4668 consensus_queue.cc:1035] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31bac92b1cd7471b8c7ff8bfa3c50df8" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 34559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:55:06.556141 4196 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53936:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:55:06.556388 4196 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53936:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:55:06.559896 4196 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8dd129ea568a4ac3a085de337681fba4: Sending DeleteTablet for 3 replicas of tablet ba7bc65e03fa461eb3ad1a1f23a80c4c
I20260619 02:55:06.561141 4317 tablet_service.cc:1514] Processing DeleteTablet for tablet ba7bc65e03fa461eb3ad1a1f23a80c4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:55:06 UTC) from {username='slave'} at 127.0.0.1:49290
I20260619 02:55:06.561138 4448 tablet_service.cc:1514] Processing DeleteTablet for tablet ba7bc65e03fa461eb3ad1a1f23a80c4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:55:06 UTC) from {username='slave'} at 127.0.0.1:57062
I20260619 02:55:06.562016 4686 tablet_replica.cc:330] T ba7bc65e03fa461eb3ad1a1f23a80c4c P dc1404916c4740778e0ba1a8b94d803e: stopping tablet replica
I20260619 02:55:06.562135 4578 tablet_service.cc:1514] Processing DeleteTablet for tablet ba7bc65e03fa461eb3ad1a1f23a80c4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:55:06 UTC) from {username='slave'} at 127.0.0.1:52178
I20260619 02:55:06.562583 4686 raft_consensus.cc:2229] T ba7bc65e03fa461eb3ad1a1f23a80c4c P dc1404916c4740778e0ba1a8b94d803e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:55:06.563050 4687 tablet_replica.cc:330] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 31bac92b1cd7471b8c7ff8bfa3c50df8: stopping tablet replica
I20260619 02:55:06.563270 4686 raft_consensus.cc:2258] T ba7bc65e03fa461eb3ad1a1f23a80c4c P dc1404916c4740778e0ba1a8b94d803e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:55:06.563448 4687 raft_consensus.cc:2229] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 31bac92b1cd7471b8c7ff8bfa3c50df8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:55:06.564159 4685 tablet_replica.cc:330] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3: stopping tablet replica
I20260619 02:55:06.563921 4687 raft_consensus.cc:2258] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 31bac92b1cd7471b8c7ff8bfa3c50df8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:55:06.564643 4685 raft_consensus.cc:2229] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:55:06.565054 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 4255
I20260619 02:55:06.565325 4685 raft_consensus.cc:2258] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:55:06.567027 4686 ts_tablet_manager.cc:1905] T ba7bc65e03fa461eb3ad1a1f23a80c4c P dc1404916c4740778e0ba1a8b94d803e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:55:06.567250 4685 ts_tablet_manager.cc:1905] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:55:06.573954 4686 ts_tablet_manager.cc:1918] T ba7bc65e03fa461eb3ad1a1f23a80c4c P dc1404916c4740778e0ba1a8b94d803e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:55:06.574133 4686 log.cc:1196] T ba7bc65e03fa461eb3ad1a1f23a80c4c P dc1404916c4740778e0ba1a8b94d803e: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-0/wal/wals/ba7bc65e03fa461eb3ad1a1f23a80c4c
I20260619 02:55:06.574694 4686 ts_tablet_manager.cc:1939] T ba7bc65e03fa461eb3ad1a1f23a80c4c P dc1404916c4740778e0ba1a8b94d803e: Deleting consensus metadata
I20260619 02:55:06.575088 4685 ts_tablet_manager.cc:1918] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:55:06.575836 4685 log.cc:1196] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-1/wal/wals/ba7bc65e03fa461eb3ad1a1f23a80c4c
I20260619 02:55:06.576361 4685 ts_tablet_manager.cc:1939] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 54f8cdc8b1b64145a4e88bdea2862fc3: Deleting consensus metadata
I20260619 02:55:06.576642 4180 catalog_manager.cc:4841] TS dc1404916c4740778e0ba1a8b94d803e (127.18.134.1:46777): tablet ba7bc65e03fa461eb3ad1a1f23a80c4c (table test-table [id=493094e9e39445baa9384c5c80d46da0]) successfully deleted
I20260619 02:55:06.578166 4181 catalog_manager.cc:4841] TS 54f8cdc8b1b64145a4e88bdea2862fc3 (127.18.134.2:40781): tablet ba7bc65e03fa461eb3ad1a1f23a80c4c (table test-table [id=493094e9e39445baa9384c5c80d46da0]) successfully deleted
I20260619 02:55:06.579429 4687 ts_tablet_manager.cc:1905] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 31bac92b1cd7471b8c7ff8bfa3c50df8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:55:06.583738 4656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:06.566590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49306 (local address 127.18.134.1:46777)
0619 02:55:06.566783 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:06.566789 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:06.575710 (+ 8921us) server_negotiation.cc:407] Connection header received
0619 02:55:06.575932 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:06.575937 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:06.576040 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:06.576117 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:06.576976 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.578046 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.580002 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.580630 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.580763 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:06.582022 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:06.582033 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:06.582050 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:06.582846 (+ 796us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:06.582905 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:06.582932 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:06.583054 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:06.583235 (+ 181us) server_negotiation.cc:299] Negotiation successful
0619 02:55:06.583363 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260619 02:55:06.584391 4377 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:06.585649 4687 ts_tablet_manager.cc:1918] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 31bac92b1cd7471b8c7ff8bfa3c50df8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:55:06.585798 4687 log.cc:1196] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 31bac92b1cd7471b8c7ff8bfa3c50df8: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781837410515220-18968-0/minicluster-data/ts-2/wal/wals/ba7bc65e03fa461eb3ad1a1f23a80c4c
I20260619 02:55:06.586321 4687 ts_tablet_manager.cc:1939] T ba7bc65e03fa461eb3ad1a1f23a80c4c P 31bac92b1cd7471b8c7ff8bfa3c50df8: Deleting consensus metadata
I20260619 02:55:06.588150 4180 catalog_manager.cc:4841] TS 31bac92b1cd7471b8c7ff8bfa3c50df8 (127.18.134.3:34559): tablet ba7bc65e03fa461eb3ad1a1f23a80c4c (table test-table [id=493094e9e39445baa9384c5c80d46da0]) successfully deleted
I20260619 02:55:06.707729 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 4255
I20260619 02:55:06.721534 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 4386
I20260619 02:55:06.728544 4658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:06.722393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57090 (local address 127.18.134.2:40781)
0619 02:55:06.722598 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:06.722604 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:06.722648 (+ 44us) server_negotiation.cc:407] Connection header received
0619 02:55:06.722712 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:06.722716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:06.722800 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:06.722888 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:06.723308 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.724082 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.725243 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.725661 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.725765 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:06.726999 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:06.727013 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:06.727033 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:06.727671 (+ 638us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:06.727728 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:06.727758 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:06.727862 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:06.728012 (+ 150us) server_negotiation.cc:299] Negotiation successful
0619 02:55:06.728119 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 02:55:06.729377 4508 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:06.837785 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 4386
I20260619 02:55:06.851984 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 4516
I20260619 02:55:06.858718 4660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:06.852683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52186 (local address 127.18.134.3:34559)
0619 02:55:06.852846 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:06.852852 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:06.852873 (+ 21us) server_negotiation.cc:407] Connection header received
0619 02:55:06.853088 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:06.853093 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:06.853190 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:06.853279 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:06.853678 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.854493 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.855812 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:06.856197 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:06.856304 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:06.857366 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:06.857376 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:06.857390 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:06.857930 (+ 540us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:06.857978 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:06.858003 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:06.858102 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:06.858271 (+ 169us) server_negotiation.cc:299] Negotiation successful
0619 02:55:06.858376 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 02:55:06.859285 4638 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:06.946120 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 4516
I20260619 02:55:06.958976 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 4165
I20260619 02:55:06.960031 4226 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:07.068238 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 4165
2026-06-19T02:55:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2721 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:55:07 dist-test-slave-9mr0 krb5kdc[4695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:55:07 dist-test-slave-9mr0 krb5kdc[4695](info): set up 2 sockets
Jun 19 02:55:07 dist-test-slave-9mr0 krb5kdc[4695](info): commencing operation
krb5kdc: starting...
W20260619 02:55:08.070564 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.965s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:55:08 dist-test-slave-9mr0 krb5kdc[4695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:55:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:55:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:55:08.306285 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:44293
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:32911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:44293
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:08.634872 4708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:08.635250 4708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:08.635388 4708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:08.642848 4708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:08.642974 4708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:08.643031 4708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:08.643118 4708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:08.643173 4708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:08.653280 4708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:32911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:44293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:44293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:08.654764 4708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:08.656225 4708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:08.668078 4713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:08.668143 4714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:08.668164 4716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:08.669107 4708 server_base.cc:1034] running on GCE node
I20260619 02:55:08.669935 4708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:08.671491 4708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:08.672713 4708 hybrid_clock.cc:648] HybridClock initialized: now 1781837708672676 us; error 59 us; skew 500 ppm
I20260619 02:55:08.673170 4708 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:08 dist-test-slave-9mr0 krb5kdc[4695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837708, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:08.676427 4708 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:55:08.677977 4708 webserver.cc:460] Webserver started at http://127.18.134.62:40591/ using document root <none> and password file <none>
I20260619 02:55:08.678761 4708 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:08.678872 4708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:08.679141 4708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:08.682226 4708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "2f9eb29cfb844c50b2e0de67c072eef0"
format_stamp: "Formatted at 2026-06-19 02:55:08 on dist-test-slave-9mr0"
server_key: "5c3d791d30d86844471ad43c312f65ef"
server_key_iv: "831567f696958bfb6a6dc75b2a96587b"
server_key_version: "encryptionkey@0"
I20260619 02:55:08.683017 4708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "2f9eb29cfb844c50b2e0de67c072eef0"
format_stamp: "Formatted at 2026-06-19 02:55:08 on dist-test-slave-9mr0"
server_key: "5c3d791d30d86844471ad43c312f65ef"
server_key_iv: "831567f696958bfb6a6dc75b2a96587b"
server_key_version: "encryptionkey@0"
I20260619 02:55:08.688556 4708 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 02:55:08.692548 4723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:08.694315 4708 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260619 02:55:08.694561 4708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "2f9eb29cfb844c50b2e0de67c072eef0"
format_stamp: "Formatted at 2026-06-19 02:55:08 on dist-test-slave-9mr0"
server_key: "5c3d791d30d86844471ad43c312f65ef"
server_key_iv: "831567f696958bfb6a6dc75b2a96587b"
server_key_version: "encryptionkey@0"
I20260619 02:55:08.694869 4708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:08.718873 4708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:08.723245 4708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:08.723546 4708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:08.741556 4708 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:44293
I20260619 02:55:08.741576 4774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:44293 every 8 connection(s)
I20260619 02:55:08.743006 4708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:55:08.746701 4775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:08.749668 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 4708
I20260619 02:55:08.749825 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:55:08.750259 18968 external_mini_cluster.cc:1428] Setting key 761753371af2426e6d30fe161b054fc5
I20260619 02:55:08.759114 4775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f9eb29cfb844c50b2e0de67c072eef0: Bootstrap starting.
I20260619 02:55:08.763234 4775 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f9eb29cfb844c50b2e0de67c072eef0: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:08.764642 4775 log.cc:824] T 00000000000000000000000000000000 P 2f9eb29cfb844c50b2e0de67c072eef0: Log is configured to *not* fsync() on all Append() calls
Jun 19 02:55:08 dist-test-slave-9mr0 krb5kdc[4695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:55:08.768441 4775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f9eb29cfb844c50b2e0de67c072eef0: No bootstrap required, opened a new log
I20260619 02:55:08.775672 4778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:08.752307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58472 (local address 127.18.134.62:44293)
0619 02:55:08.753064 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:08.753077 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:08.753126 (+ 49us) server_negotiation.cc:407] Connection header received
0619 02:55:08.754918 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:08.754944 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:08.755400 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:08.755759 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:08.757592 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:08.758978 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:08.760839 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:08.761613 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:08.761753 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:08.766882 (+ 5129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:08.766894 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:08.766915 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:08.766968 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:08.770903 (+ 3935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:08.771693 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:08.771703 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:08.771712 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:08.771930 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:08.772301 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:08.772306 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:08.772309 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:08.773024 (+ 715us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:08.773326 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:08.773703 (+ 377us) server_negotiation.cc:299] Negotiation successful
0619 02:55:08.774214 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":345,"threads_started":1}
I20260619 02:55:08.778822 4775 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f9eb29cfb844c50b2e0de67c072eef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f9eb29cfb844c50b2e0de67c072eef0" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 44293 } }
I20260619 02:55:08.779304 4775 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f9eb29cfb844c50b2e0de67c072eef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:08.779404 4775 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f9eb29cfb844c50b2e0de67c072eef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f9eb29cfb844c50b2e0de67c072eef0, State: Initialized, Role: FOLLOWER
I20260619 02:55:08.779959 4775 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f9eb29cfb844c50b2e0de67c072eef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f9eb29cfb844c50b2e0de67c072eef0" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 44293 } }
I20260619 02:55:08.780189 4775 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f9eb29cfb844c50b2e0de67c072eef0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:55:08.780320 4775 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f9eb29cfb844c50b2e0de67c072eef0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:55:08.780475 4775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f9eb29cfb844c50b2e0de67c072eef0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:08.782938 4775 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f9eb29cfb844c50b2e0de67c072eef0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f9eb29cfb844c50b2e0de67c072eef0" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 44293 } }
I20260619 02:55:08.783439 4775 leader_election.cc:302] T 00000000000000000000000000000000 P 2f9eb29cfb844c50b2e0de67c072eef0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f9eb29cfb844c50b2e0de67c072eef0; no voters:
I20260619 02:55:08.784168 4775 leader_election.cc:288] T 00000000000000000000000000000000 P 2f9eb29cfb844c50b2e0de67c072eef0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:55:08.784286 4780 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f9eb29cfb844c50b2e0de67c072eef0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:08.784935 4780 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f9eb29cfb844c50b2e0de67c072eef0 [term 1 LEADER]: Becoming Leader. State: Replica: 2f9eb29cfb844c50b2e0de67c072eef0, State: Running, Role: LEADER
I20260619 02:55:08.785516 4780 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f9eb29cfb844c50b2e0de67c072eef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f9eb29cfb844c50b2e0de67c072eef0" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 44293 } }
I20260619 02:55:08.786150 4775 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f9eb29cfb844c50b2e0de67c072eef0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:55:08.788316 4782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f9eb29cfb844c50b2e0de67c072eef0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f9eb29cfb844c50b2e0de67c072eef0. Latest consensus state: current_term: 1 leader_uuid: "2f9eb29cfb844c50b2e0de67c072eef0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f9eb29cfb844c50b2e0de67c072eef0" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 44293 } } }
I20260619 02:55:08.788630 4782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f9eb29cfb844c50b2e0de67c072eef0 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:08.789534 4786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:55:08.789340 4781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f9eb29cfb844c50b2e0de67c072eef0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f9eb29cfb844c50b2e0de67c072eef0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f9eb29cfb844c50b2e0de67c072eef0" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 44293 } } }
I20260619 02:55:08.789762 4781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f9eb29cfb844c50b2e0de67c072eef0 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:08.799516 4786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:55:08.808847 4786 catalog_manager.cc:1347] Generated new cluster ID: 6a461e5264b94fb98b1a489fb3f0a1dc
I20260619 02:55:08.808995 4786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:55:08.826956 4786 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:55:08.828851 4786 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:55:08.843470 4786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f9eb29cfb844c50b2e0de67c072eef0: Generated new TSK 0
I20260619 02:55:08.844620 4786 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.134.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
I20260619 02:55:08.906277 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:0
--local_ip_for_outbound_sockets=127.18.134.1
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:44293
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:32911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:09.246701 4801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:09.247022 4801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:09.247155 4801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:09.254570 4801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:09.254746 4801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:09.254926 4801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:55:09.265033 4801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:32911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:44293
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:09.266407 4801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:09.267915 4801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:09.281869 4806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:09.281930 4809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:09.281930 4807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:09.282999 4801 server_base.cc:1034] running on GCE node
I20260619 02:55:09.283643 4801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:09.284636 4801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:09.285867 4801 hybrid_clock.cc:648] HybridClock initialized: now 1781837709285798 us; error 57 us; skew 500 ppm
I20260619 02:55:09.286361 4801 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:09 dist-test-slave-9mr0 krb5kdc[4695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837709, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:09.289814 4801 init.cc:377] Logged in from keytab as kudu/127.18.134.1@KRBTEST.COM (short username kudu)
I20260619 02:55:09.291438 4801 webserver.cc:460] Webserver started at http://127.18.134.1:36049/ using document root <none> and password file <none>
I20260619 02:55:09.292212 4801 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:09.292326 4801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:09.292632 4801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:09.295712 4801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-0/data/instance:
uuid: "0abd0b63e2a340b0b40e68388aaa66da"
format_stamp: "Formatted at 2026-06-19 02:55:09 on dist-test-slave-9mr0"
server_key: "af2abd875f55d93f8b653a9927d73b5c"
server_key_iv: "20cdd7259f5237d45661b872cdc7d11a"
server_key_version: "encryptionkey@0"
I20260619 02:55:09.296533 4801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance:
uuid: "0abd0b63e2a340b0b40e68388aaa66da"
format_stamp: "Formatted at 2026-06-19 02:55:09 on dist-test-slave-9mr0"
server_key: "af2abd875f55d93f8b653a9927d73b5c"
server_key_iv: "20cdd7259f5237d45661b872cdc7d11a"
server_key_version: "encryptionkey@0"
I20260619 02:55:09.302609 4801 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260619 02:55:09.306843 4816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:09.308269 4801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260619 02:55:09.308439 4801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "0abd0b63e2a340b0b40e68388aaa66da"
format_stamp: "Formatted at 2026-06-19 02:55:09 on dist-test-slave-9mr0"
server_key: "af2abd875f55d93f8b653a9927d73b5c"
server_key_iv: "20cdd7259f5237d45661b872cdc7d11a"
server_key_version: "encryptionkey@0"
I20260619 02:55:09.308669 4801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:09.330654 4801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:09.335047 4801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:09.335333 4801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:09.336288 4801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:55:09.337670 4801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:55:09.337745 4801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:09.337859 4801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:55:09.337915 4801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:09.367838 4801 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:36503
I20260619 02:55:09.367918 4928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:36503 every 8 connection(s)
I20260619 02:55:09.369139 4801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:55:09.373401 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 4801
I20260619 02:55:09.373593 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance
I20260619 02:55:09.374122 18968 external_mini_cluster.cc:1428] Setting key 850097ad757ff315a14f10b30dfd1176
Jun 19 02:55:09 dist-test-slave-9mr0 krb5kdc[4695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837709, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:55:09.391835 4931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:09.371759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:45359 (local address 127.18.134.62:44293)
0619 02:55:09.372190 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:09.372194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:09.373054 (+ 860us) server_negotiation.cc:407] Connection header received
0619 02:55:09.375435 (+ 2381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:09.375440 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:09.375512 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:09.375615 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:09.377553 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:09.378256 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:09.380424 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:09.380824 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:09.380929 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:09.385040 (+ 4111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:09.385047 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:09.385052 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:09.385100 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:09.388349 (+ 3249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:09.389188 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:09.389193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:09.389196 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:09.389348 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:09.389886 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:09.389890 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:09.389893 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:09.390256 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:09.390398 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:09.391249 (+ 851us) server_negotiation.cc:299] Negotiation successful
0619 02:55:09.391491 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":219,"threads_started":1}
I20260619 02:55:09.392731 4932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:09.372193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:44293 (local address 127.18.134.1:45359)
0619 02:55:09.372883 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:09.372919 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:09.375177 (+ 2258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:09.375849 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:09.375862 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:09.376566 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:09.377361 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:09.377388 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:09.378429 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:09.378436 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:09.380229 (+ 1793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:09.380245 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:09.381049 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:09.381055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:09.381321 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:09.382126 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:55:09.382162 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:09.384760 (+ 2598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:09.388532 (+ 3772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:09.388542 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:09.388557 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:09.389050 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:09.389522 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:09.389527 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:09.389530 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:09.389762 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:09.390380 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:09.390387 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:09.390842 (+ 455us) client_negotiation.cc:769] Sending connection context
0619 02:55:09.391084 (+ 242us) client_negotiation.cc:240] Negotiation successful
0619 02:55:09.391525 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":258,"threads_started":1}
I20260619 02:55:09.394443 4929 heartbeater.cc:347] Connected to a master server at 127.18.134.62:44293
I20260619 02:55:09.394829 4929 heartbeater.cc:464] Registering TS with master...
I20260619 02:55:09.395722 4929 heartbeater.cc:511] Master 127.18.134.62:44293 requested a full tablet report, sending...
I20260619 02:55:09.398275 4740 ts_manager.cc:194] Registered new tserver with Master: 0abd0b63e2a340b0b40e68388aaa66da (127.18.134.1:36503)
WARNING: no policy specified for kudu/127.18.134.2@KRBTEST.COM; defaulting to no policy
I20260619 02:55:09.399941 4740 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.1@KRBTEST.COM'} at 127.18.134.1:45359
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
I20260619 02:55:09.430685 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:0
--local_ip_for_outbound_sockets=127.18.134.2
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:44293
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:32911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:09.753628 4935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:09.753973 4935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:09.754106 4935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:09.765146 4935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:09.765278 4935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:09.765396 4935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:55:09.775452 4935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:32911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:44293
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:09.776686 4935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:09.777969 4935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:09.790876 4940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:09.790807 4941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:09.790845 4943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:09.796171 4935 server_base.cc:1034] running on GCE node
I20260619 02:55:09.796842 4935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:09.797791 4935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:09.798983 4935 hybrid_clock.cc:648] HybridClock initialized: now 1781837709798934 us; error 41 us; skew 500 ppm
I20260619 02:55:09.799320 4935 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:09 dist-test-slave-9mr0 krb5kdc[4695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837709, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:09.802490 4935 init.cc:377] Logged in from keytab as kudu/127.18.134.2@KRBTEST.COM (short username kudu)
I20260619 02:55:09.803974 4935 webserver.cc:460] Webserver started at http://127.18.134.2:35631/ using document root <none> and password file <none>
I20260619 02:55:09.804665 4935 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:09.804781 4935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:09.805070 4935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:09.808033 4935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-1/data/instance:
uuid: "466e19fa80ce4a5badf872780835ac2e"
format_stamp: "Formatted at 2026-06-19 02:55:09 on dist-test-slave-9mr0"
server_key: "356c25d3fa0b13e69ac5dac288a259ab"
server_key_iv: "37742380fc6109710602c453c3058edf"
server_key_version: "encryptionkey@0"
I20260619 02:55:09.808796 4935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance:
uuid: "466e19fa80ce4a5badf872780835ac2e"
format_stamp: "Formatted at 2026-06-19 02:55:09 on dist-test-slave-9mr0"
server_key: "356c25d3fa0b13e69ac5dac288a259ab"
server_key_iv: "37742380fc6109710602c453c3058edf"
server_key_version: "encryptionkey@0"
I20260619 02:55:09.814103 4935 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 02:55:09.817718 4950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:09.819191 4935 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:55:09.819348 4935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "466e19fa80ce4a5badf872780835ac2e"
format_stamp: "Formatted at 2026-06-19 02:55:09 on dist-test-slave-9mr0"
server_key: "356c25d3fa0b13e69ac5dac288a259ab"
server_key_iv: "37742380fc6109710602c453c3058edf"
server_key_version: "encryptionkey@0"
I20260619 02:55:09.819553 4935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:09.833742 4935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:09.837999 4935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:09.838272 4935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:09.839303 4935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:55:09.840643 4935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:55:09.840729 4935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:09.840858 4935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:55:09.840915 4935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:09.871913 4935 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:42489
I20260619 02:55:09.871955 5062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:42489 every 8 connection(s)
I20260619 02:55:09.873167 4935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:55:09.873441 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 4935
I20260619 02:55:09.873605 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance
I20260619 02:55:09.874145 18968 external_mini_cluster.cc:1428] Setting key 1f460ff9d02139ccb0eff0e8a2887381
Jun 19 02:55:09 dist-test-slave-9mr0 krb5kdc[4695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837709, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:55:09.894757 4931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:09.875811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:46531 (local address 127.18.134.62:44293)
0619 02:55:09.875999 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:09.876004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:09.877290 (+ 1286us) server_negotiation.cc:407] Connection header received
0619 02:55:09.879740 (+ 2450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:09.879745 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:09.879818 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:09.879902 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:09.881757 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:09.882476 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:09.884563 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:09.885008 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:09.885109 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:09.888684 (+ 3575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:09.888690 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:09.888695 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:09.888744 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:09.891731 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:09.892474 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:09.892480 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:09.892483 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:09.892630 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:09.893006 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:09.893011 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:09.893014 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:09.893340 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:09.893475 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:09.894131 (+ 656us) server_negotiation.cc:299] Negotiation successful
0619 02:55:09.894371 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 02:55:09.895529 5065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:09.876331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:44293 (local address 127.18.134.2:46531)
0619 02:55:09.877119 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:09.877173 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:09.879444 (+ 2271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:09.880162 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:09.880174 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:09.880831 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:09.881596 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:09.881618 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:09.882625 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:09.882657 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:09.884405 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:09.884420 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:09.885239 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:09.885247 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:09.885479 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:09.886225 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:55:09.886250 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:09.888448 (+ 2198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:09.891867 (+ 3419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:09.891873 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:09.891887 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:09.892345 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:09.892726 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:09.892729 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:09.892732 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:09.892907 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:09.893464 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:09.893470 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:09.893792 (+ 322us) client_negotiation.cc:769] Sending connection context
0619 02:55:09.894006 (+ 214us) client_negotiation.cc:240] Negotiation successful
0619 02:55:09.894403 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":296,"threads_started":1}
I20260619 02:55:09.896996 5063 heartbeater.cc:347] Connected to a master server at 127.18.134.62:44293
I20260619 02:55:09.897311 5063 heartbeater.cc:464] Registering TS with master...
I20260619 02:55:09.898033 5063 heartbeater.cc:511] Master 127.18.134.62:44293 requested a full tablet report, sending...
I20260619 02:55:09.899489 4740 ts_manager.cc:194] Registered new tserver with Master: 466e19fa80ce4a5badf872780835ac2e (127.18.134.2:42489)
WARNING: no policy specified for kudu/127.18.134.3@KRBTEST.COM; defaulting to no policy
I20260619 02:55:09.900329 4740 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.2@KRBTEST.COM'} at 127.18.134.2:46531
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
I20260619 02:55:09.930459 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:0
--local_ip_for_outbound_sockets=127.18.134.3
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:44293
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:32911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:10.244131 5068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:10.244422 5068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:10.244556 5068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:10.251703 5068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:10.251829 5068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:10.251981 5068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:55:10.261613 5068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:32911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:44293
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:10.262938 5068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:10.264215 5068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:10.276999 5074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:10.277213 5076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:10.277444 5073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:10.277841 5068 server_base.cc:1034] running on GCE node
I20260619 02:55:10.278435 5068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:10.279400 5068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:10.280599 5068 hybrid_clock.cc:648] HybridClock initialized: now 1781837710280552 us; error 39 us; skew 500 ppm
I20260619 02:55:10.280910 5068 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:10 dist-test-slave-9mr0 krb5kdc[4695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837710, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:10.284034 5068 init.cc:377] Logged in from keytab as kudu/127.18.134.3@KRBTEST.COM (short username kudu)
I20260619 02:55:10.285550 5068 webserver.cc:460] Webserver started at http://127.18.134.3:39993/ using document root <none> and password file <none>
I20260619 02:55:10.286247 5068 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:10.286361 5068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:10.286688 5068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:10.289510 5068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-2/data/instance:
uuid: "ed6333d6051e4e609676bcf17675e134"
format_stamp: "Formatted at 2026-06-19 02:55:10 on dist-test-slave-9mr0"
server_key: "b5859db2eb179bebd907007d068b24bc"
server_key_iv: "2f146e5d4a2196e3a5ad35fe19c0c9aa"
server_key_version: "encryptionkey@0"
I20260619 02:55:10.290261 5068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance:
uuid: "ed6333d6051e4e609676bcf17675e134"
format_stamp: "Formatted at 2026-06-19 02:55:10 on dist-test-slave-9mr0"
server_key: "b5859db2eb179bebd907007d068b24bc"
server_key_iv: "2f146e5d4a2196e3a5ad35fe19c0c9aa"
server_key_version: "encryptionkey@0"
I20260619 02:55:10.295760 5068 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260619 02:55:10.299448 5083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:10.300796 5068 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:55:10.300953 5068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "ed6333d6051e4e609676bcf17675e134"
format_stamp: "Formatted at 2026-06-19 02:55:10 on dist-test-slave-9mr0"
server_key: "b5859db2eb179bebd907007d068b24bc"
server_key_iv: "2f146e5d4a2196e3a5ad35fe19c0c9aa"
server_key_version: "encryptionkey@0"
I20260619 02:55:10.301152 5068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:10.324290 5068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:10.328270 5068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:10.328545 5068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:10.329500 5068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:55:10.330951 5068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:55:10.331030 5068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:10.331163 5068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:55:10.331218 5068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:10.361027 5068 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:45159
I20260619 02:55:10.361068 5195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:45159 every 8 connection(s)
I20260619 02:55:10.362284 5068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:55:10.362592 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 5068
I20260619 02:55:10.362809 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance
I20260619 02:55:10.363384 18968 external_mini_cluster.cc:1428] Setting key 9fafb798c13db1c1f32d2a572ca10e96
Jun 19 02:55:10 dist-test-slave-9mr0 krb5kdc[4695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837710, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:55:10.383314 4931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:10.365261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:60957 (local address 127.18.134.62:44293)
0619 02:55:10.365455 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:10.365462 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:10.366735 (+ 1273us) server_negotiation.cc:407] Connection header received
0619 02:55:10.368661 (+ 1926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:10.368667 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:10.368755 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:10.368856 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:10.370400 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.371296 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.372897 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.373330 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.373455 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:10.377019 (+ 3564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:10.377025 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:10.377028 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:10.377067 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:10.380077 (+ 3010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:10.380847 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:10.380851 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:10.380854 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:10.381005 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:10.381453 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:10.381467 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:10.381469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:10.381755 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:10.381903 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:10.382688 (+ 785us) server_negotiation.cc:299] Negotiation successful
0619 02:55:10.382921 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 02:55:10.384299 5198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:10.365740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:44293 (local address 127.18.134.3:60957)
0619 02:55:10.366560 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:10.366611 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:10.368388 (+ 1777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:10.369120 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:10.369129 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:10.369650 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:10.370224 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:10.370242 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.371446 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.371452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:10.372739 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:10.372754 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.373574 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.373581 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:10.373833 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:10.374528 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:55:10.374559 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:10.376785 (+ 2226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:10.380227 (+ 3442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:10.380237 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:10.380256 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:10.380711 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:10.381121 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:10.381127 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:10.381131 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:10.381340 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:10.381896 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:10.381904 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:10.382287 (+ 383us) client_negotiation.cc:769] Sending connection context
0619 02:55:10.382559 (+ 272us) client_negotiation.cc:240] Negotiation successful
0619 02:55:10.383137 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":292,"threads_started":1}
I20260619 02:55:10.385710 5196 heartbeater.cc:347] Connected to a master server at 127.18.134.62:44293
I20260619 02:55:10.386085 5196 heartbeater.cc:464] Registering TS with master...
I20260619 02:55:10.386950 5196 heartbeater.cc:511] Master 127.18.134.62:44293 requested a full tablet report, sending...
I20260619 02:55:10.388489 4740 ts_manager.cc:194] Registered new tserver with Master: ed6333d6051e4e609676bcf17675e134 (127.18.134.3:45159)
I20260619 02:55:10.389357 4740 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.3@KRBTEST.COM'} at 127.18.134.3:60957
I20260619 02:55:10.391544 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:55:10.403281 4929 heartbeater.cc:503] Master 127.18.134.62:44293 was elected leader, sending a full tablet report...
I20260619 02:55:10.408973 4931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:10.395454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57082 (local address 127.18.134.62:44293)
0619 02:55:10.395618 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:10.395623 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:10.396049 (+ 426us) server_negotiation.cc:407] Connection header received
0619 02:55:10.396233 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:10.396237 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:10.396300 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:10.396379 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:10.397859 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.398597 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.399962 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.400386 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.400484 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:10.403011 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:10.403017 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:10.403022 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:10.403061 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:10.406035 (+ 2974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:10.406840 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:10.406844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:10.406846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:10.406970 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:10.407441 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:10.407446 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:10.407448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:10.407772 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:10.407990 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:10.408515 (+ 525us) server_negotiation.cc:299] Negotiation successful
0619 02:55:10.408661 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 02:55:10.413148 4740 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57082:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:55:10.420141 4740 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:55:10.443964 5208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:10.432627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:45159 (local address 127.0.0.1:35794)
0619 02:55:10.433325 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:10.433367 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:10.433550 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:10.436366 (+ 2816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:10.436374 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:10.436405 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:10.436953 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:10.436972 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.439002 (+ 2030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.439007 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:10.441121 (+ 2114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:10.441137 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.442888 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.442894 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:10.443091 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:10.443149 (+ 58us) client_negotiation.cc:769] Sending connection context
0619 02:55:10.443292 (+ 143us) client_negotiation.cc:240] Negotiation successful
0619 02:55:10.443489 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":151,"thread_start_us":2457,"threads_started":1}
I20260619 02:55:10.444049 5210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:10.432756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35794 (local address 127.18.134.3:45159)
0619 02:55:10.435773 (+ 3017us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:10.435783 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:10.435822 (+ 39us) server_negotiation.cc:407] Connection header received
0619 02:55:10.435924 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:10.435932 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:10.436168 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:10.436328 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:10.437210 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.438856 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.441316 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.442707 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.442840 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:10.443078 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:10.443510 (+ 432us) server_negotiation.cc:299] Negotiation successful
0619 02:55:10.443699 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1968,"thread_start_us":1774,"threads_started":1}
I20260619 02:55:10.444049 5211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:10.433282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57034 (local address 127.18.134.1:36503)
0619 02:55:10.434376 (+ 1094us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:10.434385 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:10.435508 (+ 1123us) server_negotiation.cc:407] Connection header received
0619 02:55:10.435835 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:10.435844 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:10.436131 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:10.436304 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:10.437537 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.438805 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.441649 (+ 2844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.443066 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.443195 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:10.443400 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:10.443622 (+ 222us) server_negotiation.cc:299] Negotiation successful
0619 02:55:10.443762 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":921,"thread_start_us":1498,"threads_started":1}
I20260619 02:55:10.444356 5212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:10.434450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:42489 (local address 127.0.0.1:56142)
0619 02:55:10.435646 (+ 1196us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:10.435660 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:10.435745 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:10.436724 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:10.436729 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:10.436756 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:10.437609 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:10.437622 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.439376 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.439381 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:10.441998 (+ 2617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:10.442009 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.443601 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.443608 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:10.443774 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:10.443810 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 02:55:10.443877 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 02:55:10.444004 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1074,"mutex_wait_us":47,"thread_start_us":952,"threads_started":1}
I20260619 02:55:10.444952 5209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:10.432667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:36503 (local address 127.0.0.1:57034)
0619 02:55:10.435373 (+ 2706us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:10.435394 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:10.435632 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:10.436613 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:10.436622 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:10.436647 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:10.437375 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:10.437390 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.439085 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.439093 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:10.441481 (+ 2388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:10.441495 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.443260 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.443267 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:10.443420 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:10.443452 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 02:55:10.444299 (+ 847us) client_negotiation.cc:240] Negotiation successful
0619 02:55:10.444430 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2572,"mutex_wait_us":128,"thread_start_us":967,"threads_started":1}
I20260619 02:55:10.445209 5213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:10.434912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56142 (local address 127.18.134.2:42489)
0619 02:55:10.436203 (+ 1291us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:10.436210 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:10.436237 (+ 27us) server_negotiation.cc:407] Connection header received
0619 02:55:10.436322 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:10.436329 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:10.436524 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:10.436696 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:10.437796 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.439240 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.442148 (+ 2908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.443380 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.444573 (+ 1193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:10.444708 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:10.444774 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 02:55:10.444932 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1123,"thread_start_us":1027,"threads_started":1}
I20260619 02:55:10.448081 4998 tablet_service.cc:1467] Processing CreateTablet for tablet d61e3a4d7d394bff92574b358d8e4483 (DEFAULT_TABLE table=test-table [id=4835fe840b6c4feba4f93e2a236e0953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:55:10.448238 4864 tablet_service.cc:1467] Processing CreateTablet for tablet d61e3a4d7d394bff92574b358d8e4483 (DEFAULT_TABLE table=test-table [id=4835fe840b6c4feba4f93e2a236e0953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:55:10.448238 5131 tablet_service.cc:1467] Processing CreateTablet for tablet d61e3a4d7d394bff92574b358d8e4483 (DEFAULT_TABLE table=test-table [id=4835fe840b6c4feba4f93e2a236e0953]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:55:10.449286 4998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d61e3a4d7d394bff92574b358d8e4483. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:10.449582 5131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d61e3a4d7d394bff92574b358d8e4483. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:10.449604 4864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d61e3a4d7d394bff92574b358d8e4483. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:10.461062 5214 tablet_bootstrap.cc:492] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e: Bootstrap starting.
I20260619 02:55:10.464646 5215 tablet_bootstrap.cc:492] T d61e3a4d7d394bff92574b358d8e4483 P ed6333d6051e4e609676bcf17675e134: Bootstrap starting.
I20260619 02:55:10.465382 5216 tablet_bootstrap.cc:492] T d61e3a4d7d394bff92574b358d8e4483 P 0abd0b63e2a340b0b40e68388aaa66da: Bootstrap starting.
I20260619 02:55:10.465703 5214 tablet_bootstrap.cc:654] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:10.467052 5214 log.cc:824] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:10.468708 5216 tablet_bootstrap.cc:654] T d61e3a4d7d394bff92574b358d8e4483 P 0abd0b63e2a340b0b40e68388aaa66da: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:10.469477 5215 tablet_bootstrap.cc:654] T d61e3a4d7d394bff92574b358d8e4483 P ed6333d6051e4e609676bcf17675e134: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:10.469919 5216 log.cc:824] T d61e3a4d7d394bff92574b358d8e4483 P 0abd0b63e2a340b0b40e68388aaa66da: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:10.470487 5214 tablet_bootstrap.cc:492] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e: No bootstrap required, opened a new log
I20260619 02:55:10.470525 5215 log.cc:824] T d61e3a4d7d394bff92574b358d8e4483 P ed6333d6051e4e609676bcf17675e134: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:10.470831 5214 ts_tablet_manager.cc:1397] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260619 02:55:10.477039 5216 tablet_bootstrap.cc:492] T d61e3a4d7d394bff92574b358d8e4483 P 0abd0b63e2a340b0b40e68388aaa66da: No bootstrap required, opened a new log
I20260619 02:55:10.477283 5215 tablet_bootstrap.cc:492] T d61e3a4d7d394bff92574b358d8e4483 P ed6333d6051e4e609676bcf17675e134: No bootstrap required, opened a new log
I20260619 02:55:10.477306 5216 ts_tablet_manager.cc:1397] T d61e3a4d7d394bff92574b358d8e4483 P 0abd0b63e2a340b0b40e68388aaa66da: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.005s
I20260619 02:55:10.477555 5215 ts_tablet_manager.cc:1397] T d61e3a4d7d394bff92574b358d8e4483 P ed6333d6051e4e609676bcf17675e134: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.005s
I20260619 02:55:10.481120 5214 raft_consensus.cc:348] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed6333d6051e4e609676bcf17675e134" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 45159 } } peers { permanent_uuid: "0abd0b63e2a340b0b40e68388aaa66da" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36503 } } peers { permanent_uuid: "466e19fa80ce4a5badf872780835ac2e" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42489 } }
I20260619 02:55:10.481590 5214 raft_consensus.cc:374] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:10.481734 5214 raft_consensus.cc:729] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 466e19fa80ce4a5badf872780835ac2e, State: Initialized, Role: FOLLOWER
I20260619 02:55:10.482367 5214 consensus_queue.cc:260] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed6333d6051e4e609676bcf17675e134" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 45159 } } peers { permanent_uuid: "0abd0b63e2a340b0b40e68388aaa66da" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36503 } } peers { permanent_uuid: "466e19fa80ce4a5badf872780835ac2e" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42489 } }
I20260619 02:55:10.483496 5063 heartbeater.cc:503] Master 127.18.134.62:44293 was elected leader, sending a full tablet report...
I20260619 02:55:10.485527 5214 ts_tablet_manager.cc:1428] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e: Time spent starting tablet: real 0.015s user 0.012s sys 0.000s
I20260619 02:55:10.485502 5216 raft_consensus.cc:348] T d61e3a4d7d394bff92574b358d8e4483 P 0abd0b63e2a340b0b40e68388aaa66da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed6333d6051e4e609676bcf17675e134" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 45159 } } peers { permanent_uuid: "0abd0b63e2a340b0b40e68388aaa66da" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36503 } } peers { permanent_uuid: "466e19fa80ce4a5badf872780835ac2e" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42489 } }
I20260619 02:55:10.485967 5216 raft_consensus.cc:374] T d61e3a4d7d394bff92574b358d8e4483 P 0abd0b63e2a340b0b40e68388aaa66da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:10.486088 5216 raft_consensus.cc:729] T d61e3a4d7d394bff92574b358d8e4483 P 0abd0b63e2a340b0b40e68388aaa66da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0abd0b63e2a340b0b40e68388aaa66da, State: Initialized, Role: FOLLOWER
I20260619 02:55:10.486735 5216 consensus_queue.cc:260] T d61e3a4d7d394bff92574b358d8e4483 P 0abd0b63e2a340b0b40e68388aaa66da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed6333d6051e4e609676bcf17675e134" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 45159 } } peers { permanent_uuid: "0abd0b63e2a340b0b40e68388aaa66da" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36503 } } peers { permanent_uuid: "466e19fa80ce4a5badf872780835ac2e" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42489 } }
I20260619 02:55:10.487160 5215 raft_consensus.cc:348] T d61e3a4d7d394bff92574b358d8e4483 P ed6333d6051e4e609676bcf17675e134 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed6333d6051e4e609676bcf17675e134" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 45159 } } peers { permanent_uuid: "0abd0b63e2a340b0b40e68388aaa66da" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36503 } } peers { permanent_uuid: "466e19fa80ce4a5badf872780835ac2e" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42489 } }
I20260619 02:55:10.487676 5215 raft_consensus.cc:374] T d61e3a4d7d394bff92574b358d8e4483 P ed6333d6051e4e609676bcf17675e134 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:10.487813 5215 raft_consensus.cc:729] T d61e3a4d7d394bff92574b358d8e4483 P ed6333d6051e4e609676bcf17675e134 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed6333d6051e4e609676bcf17675e134, State: Initialized, Role: FOLLOWER
I20260619 02:55:10.488438 5216 ts_tablet_manager.cc:1428] T d61e3a4d7d394bff92574b358d8e4483 P 0abd0b63e2a340b0b40e68388aaa66da: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260619 02:55:10.488369 5215 consensus_queue.cc:260] T d61e3a4d7d394bff92574b358d8e4483 P ed6333d6051e4e609676bcf17675e134 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed6333d6051e4e609676bcf17675e134" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 45159 } } peers { permanent_uuid: "0abd0b63e2a340b0b40e68388aaa66da" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36503 } } peers { permanent_uuid: "466e19fa80ce4a5badf872780835ac2e" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42489 } }
I20260619 02:55:10.489408 5196 heartbeater.cc:503] Master 127.18.134.62:44293 was elected leader, sending a full tablet report...
I20260619 02:55:10.490073 5215 ts_tablet_manager.cc:1428] T d61e3a4d7d394bff92574b358d8e4483 P ed6333d6051e4e609676bcf17675e134: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
W20260619 02:55:10.614125 5197 tablet.cc:2369] T d61e3a4d7d394bff92574b358d8e4483 P ed6333d6051e4e609676bcf17675e134: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:55:10.621402 4930 tablet.cc:2369] T d61e3a4d7d394bff92574b358d8e4483 P 0abd0b63e2a340b0b40e68388aaa66da: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 02:55:10.625078 5064 tablet.cc:2369] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:55:10.641991 5220 raft_consensus.cc:482] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:55:10.642300 5220 raft_consensus.cc:504] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed6333d6051e4e609676bcf17675e134" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 45159 } } peers { permanent_uuid: "0abd0b63e2a340b0b40e68388aaa66da" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36503 } } peers { permanent_uuid: "466e19fa80ce4a5badf872780835ac2e" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42489 } }
I20260619 02:55:10.644002 5220 leader_election.cc:288] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed6333d6051e4e609676bcf17675e134 (127.18.134.3:45159), 0abd0b63e2a340b0b40e68388aaa66da (127.18.134.1:36503)
I20260619 02:55:10.653041 5210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:10.644527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:59751 (local address 127.18.134.3:45159)
0619 02:55:10.644698 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:10.644703 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:10.645392 (+ 689us) server_negotiation.cc:407] Connection header received
0619 02:55:10.645531 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:10.645536 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:10.645609 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:10.645711 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:10.646500 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.647685 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.650553 (+ 2868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.651838 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.652020 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:10.652109 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:10.652607 (+ 498us) server_negotiation.cc:299] Negotiation successful
0619 02:55:10.652751 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 02:55:10.653081 5211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:10.644422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:36353 (local address 127.18.134.1:36503)
0619 02:55:10.644642 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:10.644648 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:10.645375 (+ 727us) server_negotiation.cc:407] Connection header received
0619 02:55:10.645518 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:10.645524 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:10.645609 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:10.645712 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:10.646400 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.647625 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.650549 (+ 2924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.651855 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.652060 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:10.652146 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:10.652607 (+ 461us) server_negotiation.cc:299] Negotiation successful
0619 02:55:10.652751 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 02:55:10.653352 5224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:10.644294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:36503 (local address 127.18.134.2:36353)
0619 02:55:10.645193 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:10.645214 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:10.645379 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:10.645728 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:10.645733 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:10.645760 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:10.646193 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:10.646211 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.647794 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.647801 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:10.650365 (+ 2564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:10.650378 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.652113 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.652124 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:10.652376 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:10.652464 (+ 88us) client_negotiation.cc:769] Sending connection context
0619 02:55:10.652576 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 02:55:10.652775 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":755,"mutex_wait_us":190,"thread_start_us":605,"threads_started":1}
I20260619 02:55:10.653373 5223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:10.644362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:45159 (local address 127.18.134.2:59751)
0619 02:55:10.645271 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:10.645290 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:10.645384 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:10.645728 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:10.645733 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:10.645821 (+ 88us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:10.646303 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:10.646313 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.647827 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.647832 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:10.650361 (+ 2529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:10.650378 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.652134 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.652139 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:10.652377 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:10.652464 (+ 87us) client_negotiation.cc:769] Sending connection context
0619 02:55:10.652559 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 02:55:10.652775 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"mutex_wait_us":232,"spinlock_wait_cycles":23424,"thread_start_us":457,"threads_started":1}
I20260619 02:55:10.654434 4884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d61e3a4d7d394bff92574b358d8e4483" candidate_uuid: "466e19fa80ce4a5badf872780835ac2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0abd0b63e2a340b0b40e68388aaa66da" is_pre_election: true
I20260619 02:55:10.654482 5151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d61e3a4d7d394bff92574b358d8e4483" candidate_uuid: "466e19fa80ce4a5badf872780835ac2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed6333d6051e4e609676bcf17675e134" is_pre_election: true
I20260619 02:55:10.654959 4884 raft_consensus.cc:2454] T d61e3a4d7d394bff92574b358d8e4483 P 0abd0b63e2a340b0b40e68388aaa66da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 466e19fa80ce4a5badf872780835ac2e in term 0.
I20260619 02:55:10.655021 5151 raft_consensus.cc:2454] T d61e3a4d7d394bff92574b358d8e4483 P ed6333d6051e4e609676bcf17675e134 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 466e19fa80ce4a5badf872780835ac2e in term 0.
I20260619 02:55:10.655666 4953 leader_election.cc:302] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0abd0b63e2a340b0b40e68388aaa66da, 466e19fa80ce4a5badf872780835ac2e, ed6333d6051e4e609676bcf17675e134; no voters:
I20260619 02:55:10.656060 5220 raft_consensus.cc:2781] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:55:10.656183 5220 raft_consensus.cc:482] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:55:10.656275 5220 raft_consensus.cc:3037] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:10.658376 5220 raft_consensus.cc:504] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed6333d6051e4e609676bcf17675e134" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 45159 } } peers { permanent_uuid: "0abd0b63e2a340b0b40e68388aaa66da" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36503 } } peers { permanent_uuid: "466e19fa80ce4a5badf872780835ac2e" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42489 } }
I20260619 02:55:10.658978 5220 leader_election.cc:288] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e [CANDIDATE]: Term 1 election: Requested vote from peers ed6333d6051e4e609676bcf17675e134 (127.18.134.3:45159), 0abd0b63e2a340b0b40e68388aaa66da (127.18.134.1:36503)
I20260619 02:55:10.659578 4884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d61e3a4d7d394bff92574b358d8e4483" candidate_uuid: "466e19fa80ce4a5badf872780835ac2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0abd0b63e2a340b0b40e68388aaa66da"
I20260619 02:55:10.659578 5151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d61e3a4d7d394bff92574b358d8e4483" candidate_uuid: "466e19fa80ce4a5badf872780835ac2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed6333d6051e4e609676bcf17675e134"
I20260619 02:55:10.659817 4884 raft_consensus.cc:3037] T d61e3a4d7d394bff92574b358d8e4483 P 0abd0b63e2a340b0b40e68388aaa66da [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:10.659817 5151 raft_consensus.cc:3037] T d61e3a4d7d394bff92574b358d8e4483 P ed6333d6051e4e609676bcf17675e134 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:10.663136 4884 raft_consensus.cc:2454] T d61e3a4d7d394bff92574b358d8e4483 P 0abd0b63e2a340b0b40e68388aaa66da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 466e19fa80ce4a5badf872780835ac2e in term 1.
I20260619 02:55:10.663136 5151 raft_consensus.cc:2454] T d61e3a4d7d394bff92574b358d8e4483 P ed6333d6051e4e609676bcf17675e134 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 466e19fa80ce4a5badf872780835ac2e in term 1.
I20260619 02:55:10.663619 4952 leader_election.cc:302] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0abd0b63e2a340b0b40e68388aaa66da, 466e19fa80ce4a5badf872780835ac2e, ed6333d6051e4e609676bcf17675e134; no voters:
I20260619 02:55:10.663976 5220 raft_consensus.cc:2781] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:10.664404 5220 raft_consensus.cc:686] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e [term 1 LEADER]: Becoming Leader. State: Replica: 466e19fa80ce4a5badf872780835ac2e, State: Running, Role: LEADER
I20260619 02:55:10.664909 5220 consensus_queue.cc:237] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed6333d6051e4e609676bcf17675e134" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 45159 } } peers { permanent_uuid: "0abd0b63e2a340b0b40e68388aaa66da" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36503 } } peers { permanent_uuid: "466e19fa80ce4a5badf872780835ac2e" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42489 } }
I20260619 02:55:10.669212 4740 catalog_manager.cc:5495] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e reported cstate change: term changed from 0 to 1, leader changed from <none> to 466e19fa80ce4a5badf872780835ac2e (127.18.134.2). New cstate: current_term: 1 leader_uuid: "466e19fa80ce4a5badf872780835ac2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed6333d6051e4e609676bcf17675e134" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 45159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0abd0b63e2a340b0b40e68388aaa66da" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "466e19fa80ce4a5badf872780835ac2e" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 42489 } health_report { overall_health: HEALTHY } } }
I20260619 02:55:10.689854 5213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:10.684499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56154 (local address 127.18.134.2:42489)
0619 02:55:10.684695 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:10.684700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:10.684718 (+ 18us) server_negotiation.cc:407] Connection header received
0619 02:55:10.684801 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:10.684805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:10.684884 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:10.685034 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:55:10.685414 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.686316 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.687864 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.688261 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.688353 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:10.688787 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:55:10.689299 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:55:10.689425 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:10.689569 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 02:55:10.689611 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 02:55:10.698184 5151 raft_consensus.cc:1261] T d61e3a4d7d394bff92574b358d8e4483 P ed6333d6051e4e609676bcf17675e134 [term 1 FOLLOWER]: Refusing update from remote peer 466e19fa80ce4a5badf872780835ac2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:55:10.698184 4884 raft_consensus.cc:1261] T d61e3a4d7d394bff92574b358d8e4483 P 0abd0b63e2a340b0b40e68388aaa66da [term 1 FOLLOWER]: Refusing update from remote peer 466e19fa80ce4a5badf872780835ac2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:55:10.699374 5225 consensus_queue.cc:1035] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0abd0b63e2a340b0b40e68388aaa66da" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:55:10.699704 5220 consensus_queue.cc:1035] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed6333d6051e4e609676bcf17675e134" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 45159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:55:10.724345 4740 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:55:10.724591 4740 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:55:10.727937 4740 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2f9eb29cfb844c50b2e0de67c072eef0: Sending DeleteTablet for 3 replicas of tablet d61e3a4d7d394bff92574b358d8e4483
I20260619 02:55:10.729108 5131 tablet_service.cc:1514] Processing DeleteTablet for tablet d61e3a4d7d394bff92574b358d8e4483 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:55:10 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:35794
I20260619 02:55:10.729123 4998 tablet_service.cc:1514] Processing DeleteTablet for tablet d61e3a4d7d394bff92574b358d8e4483 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:55:10 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:56142
I20260619 02:55:10.729262 4864 tablet_service.cc:1514] Processing DeleteTablet for tablet d61e3a4d7d394bff92574b358d8e4483 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:55:10 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:57034
I20260619 02:55:10.733170 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 4801
I20260619 02:55:10.748703 5239 tablet_replica.cc:330] T d61e3a4d7d394bff92574b358d8e4483 P 0abd0b63e2a340b0b40e68388aaa66da: stopping tablet replica
I20260619 02:55:10.749434 5239 raft_consensus.cc:2229] T d61e3a4d7d394bff92574b358d8e4483 P 0abd0b63e2a340b0b40e68388aaa66da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:55:10.750167 5239 raft_consensus.cc:2258] T d61e3a4d7d394bff92574b358d8e4483 P 0abd0b63e2a340b0b40e68388aaa66da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:55:10.752499 5239 ts_tablet_manager.cc:1905] T d61e3a4d7d394bff92574b358d8e4483 P 0abd0b63e2a340b0b40e68388aaa66da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:55:10.757332 5238 tablet_replica.cc:330] T d61e3a4d7d394bff92574b358d8e4483 P ed6333d6051e4e609676bcf17675e134: stopping tablet replica
I20260619 02:55:10.758014 5238 raft_consensus.cc:2229] T d61e3a4d7d394bff92574b358d8e4483 P ed6333d6051e4e609676bcf17675e134 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:55:10.760382 5238 raft_consensus.cc:2258] T d61e3a4d7d394bff92574b358d8e4483 P ed6333d6051e4e609676bcf17675e134 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:55:10.760460 5239 ts_tablet_manager.cc:1918] T d61e3a4d7d394bff92574b358d8e4483 P 0abd0b63e2a340b0b40e68388aaa66da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:55:10.760735 5239 log.cc:1196] T d61e3a4d7d394bff92574b358d8e4483 P 0abd0b63e2a340b0b40e68388aaa66da: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-0/wal/wals/d61e3a4d7d394bff92574b358d8e4483
I20260619 02:55:10.761339 5239 ts_tablet_manager.cc:1939] T d61e3a4d7d394bff92574b358d8e4483 P 0abd0b63e2a340b0b40e68388aaa66da: Deleting consensus metadata
I20260619 02:55:10.763497 4725 catalog_manager.cc:4841] TS 0abd0b63e2a340b0b40e68388aaa66da (127.18.134.1:36503): tablet d61e3a4d7d394bff92574b358d8e4483 (table test-table [id=4835fe840b6c4feba4f93e2a236e0953]) successfully deleted
I20260619 02:55:10.763625 5237 tablet_replica.cc:330] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e: stopping tablet replica
I20260619 02:55:10.765024 5238 ts_tablet_manager.cc:1905] T d61e3a4d7d394bff92574b358d8e4483 P ed6333d6051e4e609676bcf17675e134: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:55:10.771137 5237 raft_consensus.cc:2229] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e [term 1 LEADER]: Raft consensus shutting down.
Jun 19 02:55:10 dist-test-slave-9mr0 krb5kdc[4695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.1@KRBTEST.COM
I20260619 02:55:10.772012 5237 raft_consensus.cc:2258] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:55:10.774362 5237 ts_tablet_manager.cc:1905] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:55:10.777722 5238 ts_tablet_manager.cc:1918] T d61e3a4d7d394bff92574b358d8e4483 P ed6333d6051e4e609676bcf17675e134: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:55:10.777951 5238 log.cc:1196] T d61e3a4d7d394bff92574b358d8e4483 P ed6333d6051e4e609676bcf17675e134: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-2/wal/wals/d61e3a4d7d394bff92574b358d8e4483
I20260619 02:55:10.778528 5238 ts_tablet_manager.cc:1939] T d61e3a4d7d394bff92574b358d8e4483 P ed6333d6051e4e609676bcf17675e134: Deleting consensus metadata
I20260619 02:55:10.780520 4726 catalog_manager.cc:4841] TS ed6333d6051e4e609676bcf17675e134 (127.18.134.3:45159): tablet d61e3a4d7d394bff92574b358d8e4483 (table test-table [id=4835fe840b6c4feba4f93e2a236e0953]) successfully deleted
I20260619 02:55:10.781047 5211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:10.734431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57048 (local address 127.18.134.1:36503)
0619 02:55:10.734611 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:10.734617 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:10.751224 (+ 16607us) server_negotiation.cc:407] Connection header received
0619 02:55:10.751758 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:10.751765 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:10.751862 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:10.751943 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:10.753737 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.754937 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.758282 (+ 3345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.758988 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.759087 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:10.773172 (+ 14085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:10.773196 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:10.773215 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:10.773276 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:10.777207 (+ 3931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:10.778220 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:10.778231 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:10.778234 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:10.778424 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:10.778951 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:10.778956 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:10.778960 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:10.779470 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:10.779757 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:10.780419 (+ 662us) server_negotiation.cc:299] Negotiation successful
0619 02:55:10.780662 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 02:55:10.782121 5237 ts_tablet_manager.cc:1918] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:55:10.782176 4924 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:10.782274 5237 log.cc:1196] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781837410515220-18968-0/minicluster-data/ts-1/wal/wals/d61e3a4d7d394bff92574b358d8e4483
I20260619 02:55:10.782989 5237 ts_tablet_manager.cc:1939] T d61e3a4d7d394bff92574b358d8e4483 P 466e19fa80ce4a5badf872780835ac2e: Deleting consensus metadata
I20260619 02:55:10.785365 4725 catalog_manager.cc:4841] TS 466e19fa80ce4a5badf872780835ac2e (127.18.134.2:42489): tablet d61e3a4d7d394bff92574b358d8e4483 (table test-table [id=4835fe840b6c4feba4f93e2a236e0953]) successfully deleted
I20260619 02:55:10.870509 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 4801
I20260619 02:55:10.891814 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 4935
Jun 19 02:55:10 dist-test-slave-9mr0 krb5kdc[4695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.2@KRBTEST.COM
I20260619 02:55:10.908586 5213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:10.892568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56156 (local address 127.18.134.2:42489)
0619 02:55:10.892743 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:10.892747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:10.892764 (+ 17us) server_negotiation.cc:407] Connection header received
0619 02:55:10.892953 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:10.892957 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:10.893024 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:10.893103 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:10.894742 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.895555 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.897544 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:10.897966 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:10.898119 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:10.902487 (+ 4368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:10.902500 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:10.902515 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:10.902559 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:10.905782 (+ 3223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:10.906497 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:10.906504 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:10.906507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:10.906704 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:10.907069 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:10.907074 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:10.907078 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:10.907438 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:10.907773 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:10.908001 (+ 228us) server_negotiation.cc:299] Negotiation successful
0619 02:55:10.908229 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:55:10.909427 5058 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:11.021616 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 4935
I20260619 02:55:11.038308 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 5068
Jun 19 02:55:11 dist-test-slave-9mr0 krb5kdc[4695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.3@KRBTEST.COM
I20260619 02:55:11.052955 5210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:11.039209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35798 (local address 127.18.134.3:45159)
0619 02:55:11.039383 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:11.039389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:11.039411 (+ 22us) server_negotiation.cc:407] Connection header received
0619 02:55:11.039513 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:11.039518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:11.039589 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:11.039672 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:11.041032 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:11.041746 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:11.043065 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:11.043443 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:11.043550 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:11.047746 (+ 4196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:11.047761 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:11.047776 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:11.047811 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:11.050269 (+ 2458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:11.050952 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:11.050959 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:11.050962 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:11.051113 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:11.051474 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:11.051477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:11.051479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:11.051840 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:11.052109 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:11.052428 (+ 319us) server_negotiation.cc:299] Negotiation successful
0619 02:55:11.052642 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 02:55:11.053555 5191 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:11.154117 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 5068
I20260619 02:55:11.166868 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 4708
I20260619 02:55:11.167929 4770 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:11.275472 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 4708
2026-06-19T02:55:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4207 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-19T02:55:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:55:11Z Disabled control of system clock
I20260619 02:55:11.343454 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:38563
--webserver_interface=127.18.134.62
--webserver_port=0
--builtin_ntp_servers=127.18.134.20:40543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:38563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:55:11.665995 5248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:11.666301 5248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:11.666409 5248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:11.673964 5248 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:11.674075 5248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:11.674115 5248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:11.674194 5248 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:11.674253 5248 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:11.683661 5248 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:40543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:38563
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:38563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:11.684949 5248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:11.686722 5248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:11.698441 5253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:11.698361 5256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:11.698359 5254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:11.699213 5248 server_base.cc:1034] running on GCE node
I20260619 02:55:11.700049 5248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:11.701458 5248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:11.702713 5248 hybrid_clock.cc:648] HybridClock initialized: now 1781837711702654 us; error 37 us; skew 500 ppm
I20260619 02:55:11.703120 5248 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:55:11.705437 5248 webserver.cc:460] Webserver started at http://127.18.134.62:42331/ using document root <none> and password file <none>
I20260619 02:55:11.706123 5248 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:11.706216 5248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:11.706521 5248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:11.709481 5248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "4b081178334b44baa2933b1fa606cd8e"
format_stamp: "Formatted at 2026-06-19 02:55:11 on dist-test-slave-9mr0"
server_key: "d9c53eea51a092f9e36a042a19801755"
server_key_iv: "27090bc8ed1afceaa14f06000629982e"
server_key_version: "encryptionkey@0"
I20260619 02:55:11.710318 5248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "4b081178334b44baa2933b1fa606cd8e"
format_stamp: "Formatted at 2026-06-19 02:55:11 on dist-test-slave-9mr0"
server_key: "d9c53eea51a092f9e36a042a19801755"
server_key_iv: "27090bc8ed1afceaa14f06000629982e"
server_key_version: "encryptionkey@0"
I20260619 02:55:11.715945 5248 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 02:55:11.719664 5262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:11.721051 5248 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 02:55:11.721261 5248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "4b081178334b44baa2933b1fa606cd8e"
format_stamp: "Formatted at 2026-06-19 02:55:11 on dist-test-slave-9mr0"
server_key: "d9c53eea51a092f9e36a042a19801755"
server_key_iv: "27090bc8ed1afceaa14f06000629982e"
server_key_version: "encryptionkey@0"
I20260619 02:55:11.721464 5248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:11.736464 5248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:11.737330 5248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:11.737604 5248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:11.754663 5248 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:38563
I20260619 02:55:11.754693 5313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:38563 every 8 connection(s)
I20260619 02:55:11.755995 5248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:55:11.759703 5314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:11.765537 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 5248
I20260619 02:55:11.765722 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:55:11.766240 18968 external_mini_cluster.cc:1428] Setting key f3ef14c07b8ab8d3c9402e0033aa3d7f
I20260619 02:55:11.771057 5314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b081178334b44baa2933b1fa606cd8e: Bootstrap starting.
I20260619 02:55:11.774114 5314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b081178334b44baa2933b1fa606cd8e: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:11.775122 5314 log.cc:824] T 00000000000000000000000000000000 P 4b081178334b44baa2933b1fa606cd8e: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:11.778231 5314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b081178334b44baa2933b1fa606cd8e: No bootstrap required, opened a new log
I20260619 02:55:11.779726 5317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:11.768301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58134 (local address 127.18.134.62:38563)
0619 02:55:11.769319 (+ 1018us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:11.769331 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:11.769373 (+ 42us) server_negotiation.cc:407] Connection header received
0619 02:55:11.771142 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:11.771166 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:11.771646 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:11.772009 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:11.772578 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:11.773823 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:11.775201 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:11.776170 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:11.776355 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:11.777522 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:11.777530 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:11.777544 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:11.778067 (+ 523us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:11.778142 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:11.778163 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:11.778255 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:11.778481 (+ 226us) server_negotiation.cc:299] Negotiation successful
0619 02:55:11.778832 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":339,"threads_started":1}
I20260619 02:55:11.785581 5314 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b081178334b44baa2933b1fa606cd8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b081178334b44baa2933b1fa606cd8e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 38563 } }
I20260619 02:55:11.785928 5314 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b081178334b44baa2933b1fa606cd8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:11.786015 5314 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b081178334b44baa2933b1fa606cd8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b081178334b44baa2933b1fa606cd8e, State: Initialized, Role: FOLLOWER
I20260619 02:55:11.786662 5314 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b081178334b44baa2933b1fa606cd8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b081178334b44baa2933b1fa606cd8e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 38563 } }
I20260619 02:55:11.786991 5314 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b081178334b44baa2933b1fa606cd8e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:55:11.787120 5314 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b081178334b44baa2933b1fa606cd8e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:55:11.787282 5314 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b081178334b44baa2933b1fa606cd8e [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:11.789431 5314 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b081178334b44baa2933b1fa606cd8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b081178334b44baa2933b1fa606cd8e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 38563 } }
I20260619 02:55:11.789911 5314 leader_election.cc:302] T 00000000000000000000000000000000 P 4b081178334b44baa2933b1fa606cd8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b081178334b44baa2933b1fa606cd8e; no voters:
I20260619 02:55:11.790418 5314 leader_election.cc:288] T 00000000000000000000000000000000 P 4b081178334b44baa2933b1fa606cd8e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:55:11.790550 5319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b081178334b44baa2933b1fa606cd8e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:11.791242 5319 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b081178334b44baa2933b1fa606cd8e [term 1 LEADER]: Becoming Leader. State: Replica: 4b081178334b44baa2933b1fa606cd8e, State: Running, Role: LEADER
I20260619 02:55:11.791720 5319 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b081178334b44baa2933b1fa606cd8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b081178334b44baa2933b1fa606cd8e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 38563 } }
I20260619 02:55:11.792397 5314 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b081178334b44baa2933b1fa606cd8e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:55:11.794613 5321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b081178334b44baa2933b1fa606cd8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b081178334b44baa2933b1fa606cd8e. Latest consensus state: current_term: 1 leader_uuid: "4b081178334b44baa2933b1fa606cd8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b081178334b44baa2933b1fa606cd8e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 38563 } } }
I20260619 02:55:11.794660 5320 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b081178334b44baa2933b1fa606cd8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b081178334b44baa2933b1fa606cd8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b081178334b44baa2933b1fa606cd8e" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 38563 } } }
I20260619 02:55:11.795154 5321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b081178334b44baa2933b1fa606cd8e [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:11.795163 5320 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b081178334b44baa2933b1fa606cd8e [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:11.796743 5327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:55:11.801419 5327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:55:11.809412 5327 catalog_manager.cc:1347] Generated new cluster ID: f4d752675f5d41c299fc9dba1dae0632
I20260619 02:55:11.809532 5327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:55:11.826170 5327 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:55:11.827539 5327 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:55:11.835161 5327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b081178334b44baa2933b1fa606cd8e: Generated new TSK 0
I20260619 02:55:11.836037 5327 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:55:11.845574 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:0
--local_ip_for_outbound_sockets=127.18.134.1
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:38563
--builtin_ntp_servers=127.18.134.20:40543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:55:12.170419 5338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:12.170751 5338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:12.170894 5338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:12.178398 5338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:12.178541 5338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:12.178726 5338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:55:12.188351 5338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:40543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:38563
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:12.189648 5338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:12.191471 5338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:12.204747 5343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:12.204811 5346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:12.204787 5344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:12.205618 5338 server_base.cc:1034] running on GCE node
I20260619 02:55:12.206209 5338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:12.207161 5338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:12.208375 5338 hybrid_clock.cc:648] HybridClock initialized: now 1781837712208329 us; error 33 us; skew 500 ppm
I20260619 02:55:12.208678 5338 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:55:12.210894 5338 webserver.cc:460] Webserver started at http://127.18.134.1:43151/ using document root <none> and password file <none>
I20260619 02:55:12.211563 5338 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:12.211652 5338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:12.211966 5338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:12.214802 5338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-0/data/instance:
uuid: "37389331753d43bd9d6823d17ebc670a"
format_stamp: "Formatted at 2026-06-19 02:55:12 on dist-test-slave-9mr0"
server_key: "4b087834567b527a35ccb48143236e3b"
server_key_iv: "4a04808f56bc8601ca699121c03136d8"
server_key_version: "encryptionkey@0"
I20260619 02:55:12.215544 5338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance:
uuid: "37389331753d43bd9d6823d17ebc670a"
format_stamp: "Formatted at 2026-06-19 02:55:12 on dist-test-slave-9mr0"
server_key: "4b087834567b527a35ccb48143236e3b"
server_key_iv: "4a04808f56bc8601ca699121c03136d8"
server_key_version: "encryptionkey@0"
I20260619 02:55:12.220515 5338 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260619 02:55:12.223877 5352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:12.225251 5338 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:55:12.225394 5338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "37389331753d43bd9d6823d17ebc670a"
format_stamp: "Formatted at 2026-06-19 02:55:12 on dist-test-slave-9mr0"
server_key: "4b087834567b527a35ccb48143236e3b"
server_key_iv: "4a04808f56bc8601ca699121c03136d8"
server_key_version: "encryptionkey@0"
I20260619 02:55:12.225582 5338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:12.240224 5338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:12.241001 5338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:12.241276 5338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:12.242218 5338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:55:12.243701 5338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:55:12.243794 5338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:12.243904 5338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:55:12.243973 5338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:12.273659 5338 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:36663
I20260619 02:55:12.273707 5464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:36663 every 8 connection(s)
I20260619 02:55:12.274983 5338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:55:12.280923 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 5338
I20260619 02:55:12.281080 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance
I20260619 02:55:12.281528 18968 external_mini_cluster.cc:1428] Setting key 6122521e7c5178501fe69eab69094411
I20260619 02:55:12.284139 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:0
--local_ip_for_outbound_sockets=127.18.134.2
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:38563
--builtin_ntp_servers=127.18.134.20:40543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 02:55:12.287566 5317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:12.277453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:47333 (local address 127.18.134.62:38563)
0619 02:55:12.277628 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:12.277633 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:12.279068 (+ 1435us) server_negotiation.cc:407] Connection header received
0619 02:55:12.280896 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:12.280902 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:12.280983 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:12.281071 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:12.282154 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:12.283009 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:12.284707 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:12.285296 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:12.285418 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:12.286772 (+ 1354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:12.286779 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:12.286783 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:12.286803 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:12.286840 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:12.286855 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:12.286933 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:12.287183 (+ 250us) server_negotiation.cc:299] Negotiation successful
0619 02:55:12.287275 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 02:55:12.288435 5467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:12.277964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:38563 (local address 127.18.134.1:47333)
0619 02:55:12.278896 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:12.278948 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:12.280620 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:12.281313 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:12.281321 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:12.281407 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:12.281987 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:12.282004 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:12.283195 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:12.283201 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:12.284545 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:12.284559 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:12.285513 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:12.285519 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:12.285730 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:12.286445 (+ 715us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:55:12.286448 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:12.286486 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:55:12.286565 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:12.286952 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:12.286963 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:12.286969 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 02:55:12.287159 (+ 190us) client_negotiation.cc:240] Negotiation successful
0619 02:55:12.287437 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":252,"threads_started":1}
I20260619 02:55:12.289857 5465 heartbeater.cc:347] Connected to a master server at 127.18.134.62:38563
I20260619 02:55:12.290174 5465 heartbeater.cc:464] Registering TS with master...
I20260619 02:55:12.290918 5465 heartbeater.cc:511] Master 127.18.134.62:38563 requested a full tablet report, sending...
I20260619 02:55:12.292840 5279 ts_manager.cc:194] Registered new tserver with Master: 37389331753d43bd9d6823d17ebc670a (127.18.134.1:36663)
I20260619 02:55:12.294476 5279 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.134.1:47333
W20260619 02:55:12.599282 5468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:12.599557 5468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:12.599692 5468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:12.606930 5468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:12.607039 5468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:12.607187 5468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:55:12.616324 5468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:40543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:38563
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:12.617611 5468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:12.619477 5468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:12.633787 5474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:12.633805 5473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:12.633843 5476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:12.634555 5468 server_base.cc:1034] running on GCE node
I20260619 02:55:12.635237 5468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:12.636246 5468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:12.637467 5468 hybrid_clock.cc:648] HybridClock initialized: now 1781837712637413 us; error 43 us; skew 500 ppm
I20260619 02:55:12.637814 5468 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:55:12.640213 5468 webserver.cc:460] Webserver started at http://127.18.134.2:44851/ using document root <none> and password file <none>
I20260619 02:55:12.640924 5468 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:12.641016 5468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:12.641300 5468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:12.644317 5468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-1/data/instance:
uuid: "e6e8d4ed07ed4bd19b7eb3abb7235b6c"
format_stamp: "Formatted at 2026-06-19 02:55:12 on dist-test-slave-9mr0"
server_key: "a1af554479c74ae8ba62c496c29059f0"
server_key_iv: "cbee263249f0c73dc55a11b15f36c0b1"
server_key_version: "encryptionkey@0"
I20260619 02:55:12.645123 5468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance:
uuid: "e6e8d4ed07ed4bd19b7eb3abb7235b6c"
format_stamp: "Formatted at 2026-06-19 02:55:12 on dist-test-slave-9mr0"
server_key: "a1af554479c74ae8ba62c496c29059f0"
server_key_iv: "cbee263249f0c73dc55a11b15f36c0b1"
server_key_version: "encryptionkey@0"
I20260619 02:55:12.650502 5468 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260619 02:55:12.654105 5482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:12.655555 5468 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:55:12.655725 5468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "e6e8d4ed07ed4bd19b7eb3abb7235b6c"
format_stamp: "Formatted at 2026-06-19 02:55:12 on dist-test-slave-9mr0"
server_key: "a1af554479c74ae8ba62c496c29059f0"
server_key_iv: "cbee263249f0c73dc55a11b15f36c0b1"
server_key_version: "encryptionkey@0"
I20260619 02:55:12.655937 5468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:12.679080 5468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:12.679970 5468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:12.680275 5468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:12.681257 5468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:55:12.682909 5468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:55:12.683009 5468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:12.683122 5468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:55:12.683204 5468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:12.713936 5468 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:46145
I20260619 02:55:12.714012 5594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:46145 every 8 connection(s)
I20260619 02:55:12.715269 5468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:55:12.717866 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 5468
I20260619 02:55:12.718032 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance
I20260619 02:55:12.718452 18968 external_mini_cluster.cc:1428] Setting key 8b857f6e53ed60c29048eebce8ba73da
I20260619 02:55:12.721565 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:0
--local_ip_for_outbound_sockets=127.18.134.3
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:38563
--builtin_ntp_servers=127.18.134.20:40543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 02:55:12.729588 5317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:12.717843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:57271 (local address 127.18.134.62:38563)
0619 02:55:12.718184 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:12.718191 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:12.719790 (+ 1599us) server_negotiation.cc:407] Connection header received
0619 02:55:12.722129 (+ 2339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:12.722133 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:12.722199 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:12.722282 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:12.723678 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:12.724489 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:12.726648 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:12.727083 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:12.727188 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:12.728826 (+ 1638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:12.728832 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:12.728835 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:12.728855 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:12.728888 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:12.728899 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:12.728989 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:12.729208 (+ 219us) server_negotiation.cc:299] Negotiation successful
0619 02:55:12.729303 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192}
I20260619 02:55:12.730762 5597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:12.718498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:38563 (local address 127.18.134.2:57271)
0619 02:55:12.719645 (+ 1147us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:12.719689 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:12.721822 (+ 2133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:12.722546 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:12.722558 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:12.722713 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:12.723497 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:12.723519 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:12.724672 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:12.724677 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:12.726472 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:12.726488 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:12.727305 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:12.727312 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:12.727592 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:12.728419 (+ 827us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:55:12.728423 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:12.728480 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:55:12.728603 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:12.729001 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:12.729010 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:12.729018 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 02:55:12.729200 (+ 182us) client_negotiation.cc:240] Negotiation successful
0619 02:55:12.729637 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":397,"threads_started":1}
I20260619 02:55:12.732268 5595 heartbeater.cc:347] Connected to a master server at 127.18.134.62:38563
I20260619 02:55:12.732609 5595 heartbeater.cc:464] Registering TS with master...
I20260619 02:55:12.733363 5595 heartbeater.cc:511] Master 127.18.134.62:38563 requested a full tablet report, sending...
I20260619 02:55:12.734870 5279 ts_manager.cc:194] Registered new tserver with Master: e6e8d4ed07ed4bd19b7eb3abb7235b6c (127.18.134.2:46145)
I20260619 02:55:12.735702 5279 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.134.2:57271
W20260619 02:55:13.047622 5598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:13.047925 5598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:13.048048 5598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:13.055735 5598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:13.055886 5598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:13.056095 5598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:55:13.068015 5598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:40543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:38563
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:13.069257 5598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:13.071059 5598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:13.084434 5606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:13.084539 5603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:13.084476 5604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:13.085565 5598 server_base.cc:1034] running on GCE node
I20260619 02:55:13.086261 5598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:13.087298 5598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:13.088490 5598 hybrid_clock.cc:648] HybridClock initialized: now 1781837713088432 us; error 53 us; skew 500 ppm
I20260619 02:55:13.088809 5598 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:55:13.091125 5598 webserver.cc:460] Webserver started at http://127.18.134.3:35775/ using document root <none> and password file <none>
I20260619 02:55:13.091816 5598 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:13.091909 5598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:13.092247 5598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:13.095342 5598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-2/data/instance:
uuid: "7f1672c77bc8423cb16c264dff33d153"
format_stamp: "Formatted at 2026-06-19 02:55:13 on dist-test-slave-9mr0"
server_key: "7c5877100e138e4c6f46816cee8f602c"
server_key_iv: "b778c78ad72e875bafa34440dec6226a"
server_key_version: "encryptionkey@0"
I20260619 02:55:13.096171 5598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance:
uuid: "7f1672c77bc8423cb16c264dff33d153"
format_stamp: "Formatted at 2026-06-19 02:55:13 on dist-test-slave-9mr0"
server_key: "7c5877100e138e4c6f46816cee8f602c"
server_key_iv: "b778c78ad72e875bafa34440dec6226a"
server_key_version: "encryptionkey@0"
I20260619 02:55:13.101768 5598 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 02:55:13.105443 5612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:13.107009 5598 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:55:13.107188 5598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "7f1672c77bc8423cb16c264dff33d153"
format_stamp: "Formatted at 2026-06-19 02:55:13 on dist-test-slave-9mr0"
server_key: "7c5877100e138e4c6f46816cee8f602c"
server_key_iv: "b778c78ad72e875bafa34440dec6226a"
server_key_version: "encryptionkey@0"
I20260619 02:55:13.107399 5598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:13.127821 5598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:13.128741 5598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:13.129038 5598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:13.130105 5598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:55:13.131647 5598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:55:13.131744 5598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:13.131850 5598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:55:13.131910 5598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:13.160262 5598 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:43741
I20260619 02:55:13.160337 5724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:43741 every 8 connection(s)
I20260619 02:55:13.161567 5598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:55:13.167838 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 5598
I20260619 02:55:13.168025 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance
I20260619 02:55:13.168445 18968 external_mini_cluster.cc:1428] Setting key 56725d3a2439a466456cab46c4a54a06
I20260619 02:55:13.174402 5317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:13.164232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:33583 (local address 127.18.134.62:38563)
0619 02:55:13.164465 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:13.164471 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:13.165833 (+ 1362us) server_negotiation.cc:407] Connection header received
0619 02:55:13.167637 (+ 1804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:13.167643 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:13.167734 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:13.167857 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:13.169014 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.169823 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.171779 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.172186 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.172282 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:13.173545 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:13.173550 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:13.173553 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:13.173571 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:13.173603 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:13.173615 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:13.173694 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:13.174079 (+ 385us) server_negotiation.cc:299] Negotiation successful
0619 02:55:13.174171 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260619 02:55:13.175515 5727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:13.164769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:38563 (local address 127.18.134.3:33583)
0619 02:55:13.165660 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:13.165705 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:13.167337 (+ 1632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:13.168065 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:13.168077 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:13.168208 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:13.168828 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:13.168846 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.169975 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.169979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:13.171650 (+ 1671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:13.171662 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.172363 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.172368 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:13.172572 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:13.173218 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:55:13.173221 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:13.173263 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:55:13.173353 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:13.173750 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:13.173760 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:13.173767 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 02:55:13.174029 (+ 262us) client_negotiation.cc:240] Negotiation successful
0619 02:55:13.174389 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":264,"threads_started":1}
I20260619 02:55:13.177031 5725 heartbeater.cc:347] Connected to a master server at 127.18.134.62:38563
I20260619 02:55:13.177333 5725 heartbeater.cc:464] Registering TS with master...
I20260619 02:55:13.178058 5725 heartbeater.cc:511] Master 127.18.134.62:38563 requested a full tablet report, sending...
I20260619 02:55:13.179605 5279 ts_manager.cc:194] Registered new tserver with Master: 7f1672c77bc8423cb16c264dff33d153 (127.18.134.3:43741)
I20260619 02:55:13.180409 5279 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.134.3:33583
I20260619 02:55:13.183652 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:55:13.193686 5317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:13.187514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58144 (local address 127.18.134.62:38563)
0619 02:55:13.187726 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:13.187733 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:13.188050 (+ 317us) server_negotiation.cc:407] Connection header received
0619 02:55:13.188214 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:13.188218 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:13.188283 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:13.188366 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:13.188879 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.189623 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.191195 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.191605 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.191721 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:13.192805 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:13.192810 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:13.192814 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:13.192834 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:13.192869 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:13.192881 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:13.192993 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:13.193292 (+ 299us) server_negotiation.cc:299] Negotiation successful
0619 02:55:13.193356 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 02:55:13.197611 5279 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:55:13.204457 5279 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:55:13.228184 5737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:13.217340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33380 (local address 127.18.134.2:46145)
0619 02:55:13.217892 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:13.217901 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:13.219152 (+ 1251us) server_negotiation.cc:407] Connection header received
0619 02:55:13.219254 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:13.219263 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:13.219522 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:13.219714 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:13.221329 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.222545 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.225777 (+ 3232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.226862 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.226945 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:13.227165 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:13.227649 (+ 484us) server_negotiation.cc:299] Negotiation successful
0619 02:55:13.227817 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":508,"threads_started":1}
I20260619 02:55:13.228237 5740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:13.216845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:46145 (local address 127.0.0.1:33380)
0619 02:55:13.218775 (+ 1930us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:13.218824 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:13.219006 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:13.220040 (+ 1034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:13.220048 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:13.220094 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:13.221118 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:13.221131 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.222735 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.222740 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:13.225533 (+ 2793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:13.225552 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.227295 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.227299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:13.227436 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:13.227471 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 02:55:13.227581 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 02:55:13.227799 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1555,"mutex_wait_us":663,"spinlock_wait_cycles":8832,"thread_start_us":1484,"threads_started":1}
I20260619 02:55:13.228705 5738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:13.217376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44726 (local address 127.18.134.1:36663)
0619 02:55:13.217881 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:13.217889 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:13.218940 (+ 1051us) server_negotiation.cc:407] Connection header received
0619 02:55:13.219278 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:13.219287 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:13.219559 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:13.219738 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:13.220909 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.222582 (+ 1673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.225826 (+ 3244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.227262 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.227365 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:13.227599 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:13.228195 (+ 596us) server_negotiation.cc:299] Negotiation successful
0619 02:55:13.228360 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":266,"threads_started":1}
I20260619 02:55:13.229511 5739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:13.216871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:36663 (local address 127.0.0.1:44726)
0619 02:55:13.218776 (+ 1905us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:13.218820 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:13.219007 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:13.220041 (+ 1034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:13.220048 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:13.220099 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:13.220680 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:13.220694 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.222818 (+ 2124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.222824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:13.225580 (+ 2756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:13.225594 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.227769 (+ 2175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.227776 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:13.227964 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:13.227997 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 02:55:13.228974 (+ 977us) client_negotiation.cc:240] Negotiation successful
0619 02:55:13.229115 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1779,"mutex_wait_us":330,"spinlock_wait_cycles":44544,"thread_start_us":2113,"threads_started":1}
I20260619 02:55:13.232821 5742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:13.219697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44728 (local address 127.18.134.3:43741)
0619 02:55:13.220641 (+ 944us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:13.220649 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:13.224082 (+ 3433us) server_negotiation.cc:407] Connection header received
0619 02:55:13.224247 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:13.224257 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:13.224525 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:13.224734 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:13.225853 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.227408 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.230345 (+ 2937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.231616 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.231762 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:13.232029 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:13.232288 (+ 259us) server_negotiation.cc:299] Negotiation successful
0619 02:55:13.232464 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":792,"thread_start_us":633,"threads_started":1}
I20260619 02:55:13.232786 5530 tablet_service.cc:1467] Processing CreateTablet for tablet b795eee02f1942c19f5e2d9379cce1a0 (DEFAULT_TABLE table=test-table [id=35ccf4e34b394436bb92af3b861da4ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:55:13.233565 5741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:13.219381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:43741 (local address 127.0.0.1:44728)
0619 02:55:13.223932 (+ 4551us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:13.223959 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:13.224099 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:13.225017 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:13.225022 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:13.225052 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:13.225673 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:13.225682 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.227583 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.227588 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:13.230142 (+ 2554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:13.230156 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.231802 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.231808 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:13.231985 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:13.232027 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 02:55:13.233091 (+ 1064us) client_negotiation.cc:240] Negotiation successful
0619 02:55:13.233234 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4413,"mutex_wait_us":3,"thread_start_us":4267,"threads_started":1}
I20260619 02:55:13.234321 5530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b795eee02f1942c19f5e2d9379cce1a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:13.234139 5400 tablet_service.cc:1467] Processing CreateTablet for tablet b795eee02f1942c19f5e2d9379cce1a0 (DEFAULT_TABLE table=test-table [id=35ccf4e34b394436bb92af3b861da4ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:55:13.235549 5400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b795eee02f1942c19f5e2d9379cce1a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:13.237382 5660 tablet_service.cc:1467] Processing CreateTablet for tablet b795eee02f1942c19f5e2d9379cce1a0 (DEFAULT_TABLE table=test-table [id=35ccf4e34b394436bb92af3b861da4ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:55:13.238834 5660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b795eee02f1942c19f5e2d9379cce1a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:13.246870 5743 tablet_bootstrap.cc:492] T b795eee02f1942c19f5e2d9379cce1a0 P e6e8d4ed07ed4bd19b7eb3abb7235b6c: Bootstrap starting.
I20260619 02:55:13.250002 5744 tablet_bootstrap.cc:492] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a: Bootstrap starting.
I20260619 02:55:13.250245 5743 tablet_bootstrap.cc:654] T b795eee02f1942c19f5e2d9379cce1a0 P e6e8d4ed07ed4bd19b7eb3abb7235b6c: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:13.251636 5743 log.cc:824] T b795eee02f1942c19f5e2d9379cce1a0 P e6e8d4ed07ed4bd19b7eb3abb7235b6c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:13.253608 5744 tablet_bootstrap.cc:654] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:13.254736 5746 tablet_bootstrap.cc:492] T b795eee02f1942c19f5e2d9379cce1a0 P 7f1672c77bc8423cb16c264dff33d153: Bootstrap starting.
I20260619 02:55:13.254905 5744 log.cc:824] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:13.256572 5743 tablet_bootstrap.cc:492] T b795eee02f1942c19f5e2d9379cce1a0 P e6e8d4ed07ed4bd19b7eb3abb7235b6c: No bootstrap required, opened a new log
I20260619 02:55:13.256808 5743 ts_tablet_manager.cc:1397] T b795eee02f1942c19f5e2d9379cce1a0 P e6e8d4ed07ed4bd19b7eb3abb7235b6c: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260619 02:55:13.258195 5744 tablet_bootstrap.cc:492] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a: No bootstrap required, opened a new log
I20260619 02:55:13.258227 5746 tablet_bootstrap.cc:654] T b795eee02f1942c19f5e2d9379cce1a0 P 7f1672c77bc8423cb16c264dff33d153: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:13.258466 5744 ts_tablet_manager.cc:1397] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260619 02:55:13.259474 5746 log.cc:824] T b795eee02f1942c19f5e2d9379cce1a0 P 7f1672c77bc8423cb16c264dff33d153: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:13.262884 5746 tablet_bootstrap.cc:492] T b795eee02f1942c19f5e2d9379cce1a0 P 7f1672c77bc8423cb16c264dff33d153: No bootstrap required, opened a new log
I20260619 02:55:13.263188 5746 ts_tablet_manager.cc:1397] T b795eee02f1942c19f5e2d9379cce1a0 P 7f1672c77bc8423cb16c264dff33d153: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260619 02:55:13.265086 5743 raft_consensus.cc:348] T b795eee02f1942c19f5e2d9379cce1a0 P e6e8d4ed07ed4bd19b7eb3abb7235b6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6e8d4ed07ed4bd19b7eb3abb7235b6c" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 46145 } } peers { permanent_uuid: "37389331753d43bd9d6823d17ebc670a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36663 } } peers { permanent_uuid: "7f1672c77bc8423cb16c264dff33d153" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43741 } }
I20260619 02:55:13.265458 5743 raft_consensus.cc:374] T b795eee02f1942c19f5e2d9379cce1a0 P e6e8d4ed07ed4bd19b7eb3abb7235b6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:13.265547 5743 raft_consensus.cc:729] T b795eee02f1942c19f5e2d9379cce1a0 P e6e8d4ed07ed4bd19b7eb3abb7235b6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6e8d4ed07ed4bd19b7eb3abb7235b6c, State: Initialized, Role: FOLLOWER
I20260619 02:55:13.266147 5743 consensus_queue.cc:260] T b795eee02f1942c19f5e2d9379cce1a0 P e6e8d4ed07ed4bd19b7eb3abb7235b6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6e8d4ed07ed4bd19b7eb3abb7235b6c" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 46145 } } peers { permanent_uuid: "37389331753d43bd9d6823d17ebc670a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36663 } } peers { permanent_uuid: "7f1672c77bc8423cb16c264dff33d153" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43741 } }
I20260619 02:55:13.267220 5595 heartbeater.cc:503] Master 127.18.134.62:38563 was elected leader, sending a full tablet report...
I20260619 02:55:13.268494 5744 raft_consensus.cc:348] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6e8d4ed07ed4bd19b7eb3abb7235b6c" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 46145 } } peers { permanent_uuid: "37389331753d43bd9d6823d17ebc670a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36663 } } peers { permanent_uuid: "7f1672c77bc8423cb16c264dff33d153" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43741 } }
I20260619 02:55:13.268965 5744 raft_consensus.cc:374] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:13.269057 5744 raft_consensus.cc:729] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37389331753d43bd9d6823d17ebc670a, State: Initialized, Role: FOLLOWER
I20260619 02:55:13.269124 5743 ts_tablet_manager.cc:1428] T b795eee02f1942c19f5e2d9379cce1a0 P e6e8d4ed07ed4bd19b7eb3abb7235b6c: Time spent starting tablet: real 0.012s user 0.009s sys 0.003s
I20260619 02:55:13.269836 5744 consensus_queue.cc:260] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6e8d4ed07ed4bd19b7eb3abb7235b6c" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 46145 } } peers { permanent_uuid: "37389331753d43bd9d6823d17ebc670a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36663 } } peers { permanent_uuid: "7f1672c77bc8423cb16c264dff33d153" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43741 } }
I20260619 02:55:13.271072 5465 heartbeater.cc:503] Master 127.18.134.62:38563 was elected leader, sending a full tablet report...
I20260619 02:55:13.271654 5744 ts_tablet_manager.cc:1428] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
I20260619 02:55:13.274580 5746 raft_consensus.cc:348] T b795eee02f1942c19f5e2d9379cce1a0 P 7f1672c77bc8423cb16c264dff33d153 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6e8d4ed07ed4bd19b7eb3abb7235b6c" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 46145 } } peers { permanent_uuid: "37389331753d43bd9d6823d17ebc670a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36663 } } peers { permanent_uuid: "7f1672c77bc8423cb16c264dff33d153" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43741 } }
I20260619 02:55:13.275028 5746 raft_consensus.cc:374] T b795eee02f1942c19f5e2d9379cce1a0 P 7f1672c77bc8423cb16c264dff33d153 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:13.275148 5746 raft_consensus.cc:729] T b795eee02f1942c19f5e2d9379cce1a0 P 7f1672c77bc8423cb16c264dff33d153 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f1672c77bc8423cb16c264dff33d153, State: Initialized, Role: FOLLOWER
I20260619 02:55:13.275840 5746 consensus_queue.cc:260] T b795eee02f1942c19f5e2d9379cce1a0 P 7f1672c77bc8423cb16c264dff33d153 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6e8d4ed07ed4bd19b7eb3abb7235b6c" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 46145 } } peers { permanent_uuid: "37389331753d43bd9d6823d17ebc670a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36663 } } peers { permanent_uuid: "7f1672c77bc8423cb16c264dff33d153" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43741 } }
I20260619 02:55:13.276962 5725 heartbeater.cc:503] Master 127.18.134.62:38563 was elected leader, sending a full tablet report...
W20260619 02:55:13.277079 5466 tablet.cc:2369] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:55:13.277576 5746 ts_tablet_manager.cc:1428] T b795eee02f1942c19f5e2d9379cce1a0 P 7f1672c77bc8423cb16c264dff33d153: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260619 02:55:13.356326 5750 raft_consensus.cc:482] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:55:13.356670 5750 raft_consensus.cc:504] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6e8d4ed07ed4bd19b7eb3abb7235b6c" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 46145 } } peers { permanent_uuid: "37389331753d43bd9d6823d17ebc670a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36663 } } peers { permanent_uuid: "7f1672c77bc8423cb16c264dff33d153" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43741 } }
I20260619 02:55:13.358436 5750 leader_election.cc:288] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6e8d4ed07ed4bd19b7eb3abb7235b6c (127.18.134.2:46145), 7f1672c77bc8423cb16c264dff33d153 (127.18.134.3:43741)
I20260619 02:55:13.367861 5737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:13.358905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:59851 (local address 127.18.134.2:46145)
0619 02:55:13.359159 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:13.359165 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:13.359899 (+ 734us) server_negotiation.cc:407] Connection header received
0619 02:55:13.360166 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:13.360171 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:13.360256 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:13.360352 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:13.361043 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.362142 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.365309 (+ 3167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.366604 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.366820 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:13.366896 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:13.367457 (+ 561us) server_negotiation.cc:299] Negotiation successful
0619 02:55:13.367627 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 02:55:13.367861 5742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:13.359052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:58711 (local address 127.18.134.3:43741)
0619 02:55:13.359242 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:13.359248 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:13.360105 (+ 857us) server_negotiation.cc:407] Connection header received
0619 02:55:13.360247 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:13.360252 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:13.360324 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:13.360401 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:13.361065 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.362142 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.365267 (+ 3125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.366598 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.366820 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:13.366896 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:13.367467 (+ 571us) server_negotiation.cc:299] Negotiation successful
0619 02:55:13.367614 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 02:55:13.367973 5752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:13.358919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:46145 (local address 127.18.134.1:59851)
0619 02:55:13.359781 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:13.359810 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:13.359979 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:13.360386 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:13.360391 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:13.360419 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:13.360891 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:13.360908 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.362288 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.362294 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:13.364960 (+ 2666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:13.364982 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.366877 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.366885 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:13.367206 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:13.367279 (+ 73us) client_negotiation.cc:769] Sending connection context
0619 02:55:13.367382 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 02:55:13.367564 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"mutex_wait_us":333,"thread_start_us":397,"threads_started":1}
I20260619 02:55:13.367996 5753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:13.358919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:43741 (local address 127.18.134.1:58711)
0619 02:55:13.359975 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:13.359989 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:13.360119 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:13.360417 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:13.360420 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:13.360437 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:13.360920 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:13.360930 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.362288 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.362294 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:13.364959 (+ 2665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:13.364982 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.366891 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.366896 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:13.367210 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:13.367274 (+ 64us) client_negotiation.cc:769] Sending connection context
0619 02:55:13.367382 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 02:55:13.367552 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":941,"mutex_wait_us":537,"thread_start_us":456,"threads_started":1}
I20260619 02:55:13.369499 5550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b795eee02f1942c19f5e2d9379cce1a0" candidate_uuid: "37389331753d43bd9d6823d17ebc670a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6e8d4ed07ed4bd19b7eb3abb7235b6c" is_pre_election: true
I20260619 02:55:13.369534 5680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b795eee02f1942c19f5e2d9379cce1a0" candidate_uuid: "37389331753d43bd9d6823d17ebc670a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f1672c77bc8423cb16c264dff33d153" is_pre_election: true
I20260619 02:55:13.370110 5550 raft_consensus.cc:2454] T b795eee02f1942c19f5e2d9379cce1a0 P e6e8d4ed07ed4bd19b7eb3abb7235b6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37389331753d43bd9d6823d17ebc670a in term 0.
I20260619 02:55:13.370110 5680 raft_consensus.cc:2454] T b795eee02f1942c19f5e2d9379cce1a0 P 7f1672c77bc8423cb16c264dff33d153 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37389331753d43bd9d6823d17ebc670a in term 0.
I20260619 02:55:13.370898 5356 leader_election.cc:302] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37389331753d43bd9d6823d17ebc670a, e6e8d4ed07ed4bd19b7eb3abb7235b6c; no voters:
I20260619 02:55:13.371402 5750 raft_consensus.cc:2781] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:55:13.371552 5750 raft_consensus.cc:482] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:55:13.371641 5750 raft_consensus.cc:3037] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:13.373832 5750 raft_consensus.cc:504] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6e8d4ed07ed4bd19b7eb3abb7235b6c" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 46145 } } peers { permanent_uuid: "37389331753d43bd9d6823d17ebc670a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36663 } } peers { permanent_uuid: "7f1672c77bc8423cb16c264dff33d153" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43741 } }
I20260619 02:55:13.374492 5750 leader_election.cc:288] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a [CANDIDATE]: Term 1 election: Requested vote from peers e6e8d4ed07ed4bd19b7eb3abb7235b6c (127.18.134.2:46145), 7f1672c77bc8423cb16c264dff33d153 (127.18.134.3:43741)
I20260619 02:55:13.375190 5550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b795eee02f1942c19f5e2d9379cce1a0" candidate_uuid: "37389331753d43bd9d6823d17ebc670a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6e8d4ed07ed4bd19b7eb3abb7235b6c"
I20260619 02:55:13.375187 5680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b795eee02f1942c19f5e2d9379cce1a0" candidate_uuid: "37389331753d43bd9d6823d17ebc670a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f1672c77bc8423cb16c264dff33d153"
I20260619 02:55:13.375444 5550 raft_consensus.cc:3037] T b795eee02f1942c19f5e2d9379cce1a0 P e6e8d4ed07ed4bd19b7eb3abb7235b6c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:13.375448 5680 raft_consensus.cc:3037] T b795eee02f1942c19f5e2d9379cce1a0 P 7f1672c77bc8423cb16c264dff33d153 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:13.378921 5680 raft_consensus.cc:2454] T b795eee02f1942c19f5e2d9379cce1a0 P 7f1672c77bc8423cb16c264dff33d153 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37389331753d43bd9d6823d17ebc670a in term 1.
I20260619 02:55:13.378968 5550 raft_consensus.cc:2454] T b795eee02f1942c19f5e2d9379cce1a0 P e6e8d4ed07ed4bd19b7eb3abb7235b6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37389331753d43bd9d6823d17ebc670a in term 1.
I20260619 02:55:13.379470 5356 leader_election.cc:302] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 37389331753d43bd9d6823d17ebc670a, 7f1672c77bc8423cb16c264dff33d153, e6e8d4ed07ed4bd19b7eb3abb7235b6c; no voters:
I20260619 02:55:13.379838 5750 raft_consensus.cc:2781] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:13.380293 5750 raft_consensus.cc:686] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a [term 1 LEADER]: Becoming Leader. State: Replica: 37389331753d43bd9d6823d17ebc670a, State: Running, Role: LEADER
I20260619 02:55:13.380805 5750 consensus_queue.cc:237] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6e8d4ed07ed4bd19b7eb3abb7235b6c" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 46145 } } peers { permanent_uuid: "37389331753d43bd9d6823d17ebc670a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36663 } } peers { permanent_uuid: "7f1672c77bc8423cb16c264dff33d153" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43741 } }
I20260619 02:55:13.385602 5277 catalog_manager.cc:5495] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a reported cstate change: term changed from 0 to 1, leader changed from <none> to 37389331753d43bd9d6823d17ebc670a (127.18.134.1). New cstate: current_term: 1 leader_uuid: "37389331753d43bd9d6823d17ebc670a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6e8d4ed07ed4bd19b7eb3abb7235b6c" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 46145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37389331753d43bd9d6823d17ebc670a" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 36663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f1672c77bc8423cb16c264dff33d153" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43741 } health_report { overall_health: UNKNOWN } } }
W20260619 02:55:13.413337 5726 tablet.cc:2369] T b795eee02f1942c19f5e2d9379cce1a0 P 7f1672c77bc8423cb16c264dff33d153: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:55:13.427944 5738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:13.422378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44734 (local address 127.18.134.1:36663)
0619 02:55:13.422607 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:13.422614 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:13.422708 (+ 94us) server_negotiation.cc:407] Connection header received
0619 02:55:13.422936 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:13.422940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:13.423038 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:13.423202 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:55:13.423578 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.424260 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.425668 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.426135 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.426245 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:13.426845 (+ 600us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:55:13.427356 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:55:13.427512 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:13.427644 (+ 132us) server_negotiation.cc:299] Negotiation successful
0619 02:55:13.427688 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260619 02:55:13.436059 5680 raft_consensus.cc:1261] T b795eee02f1942c19f5e2d9379cce1a0 P 7f1672c77bc8423cb16c264dff33d153 [term 1 FOLLOWER]: Refusing update from remote peer 37389331753d43bd9d6823d17ebc670a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:55:13.436084 5550 raft_consensus.cc:1261] T b795eee02f1942c19f5e2d9379cce1a0 P e6e8d4ed07ed4bd19b7eb3abb7235b6c [term 1 FOLLOWER]: Refusing update from remote peer 37389331753d43bd9d6823d17ebc670a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:55:13.437114 5754 consensus_queue.cc:1035] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f1672c77bc8423cb16c264dff33d153" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:55:13.437532 5750 consensus_queue.cc:1035] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6e8d4ed07ed4bd19b7eb3abb7235b6c" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 46145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:55:13.463515 5277 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:55:13.463778 5277 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:55:13.468640 5277 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4b081178334b44baa2933b1fa606cd8e: Sending DeleteTablet for 3 replicas of tablet b795eee02f1942c19f5e2d9379cce1a0
I20260619 02:55:13.469938 5530 tablet_service.cc:1514] Processing DeleteTablet for tablet b795eee02f1942c19f5e2d9379cce1a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:55:13 UTC) from {username='slave'} at 127.0.0.1:33380
I20260619 02:55:13.470208 5400 tablet_service.cc:1514] Processing DeleteTablet for tablet b795eee02f1942c19f5e2d9379cce1a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:55:13 UTC) from {username='slave'} at 127.0.0.1:44726
I20260619 02:55:13.470417 5660 tablet_service.cc:1514] Processing DeleteTablet for tablet b795eee02f1942c19f5e2d9379cce1a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:55:13 UTC) from {username='slave'} at 127.0.0.1:44728
I20260619 02:55:13.472361 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 5338
I20260619 02:55:13.485294 5767 tablet_replica.cc:330] T b795eee02f1942c19f5e2d9379cce1a0 P e6e8d4ed07ed4bd19b7eb3abb7235b6c: stopping tablet replica
I20260619 02:55:13.485435 5768 tablet_replica.cc:330] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a: stopping tablet replica
I20260619 02:55:13.485826 5767 raft_consensus.cc:2229] T b795eee02f1942c19f5e2d9379cce1a0 P e6e8d4ed07ed4bd19b7eb3abb7235b6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:55:13.485872 5768 raft_consensus.cc:2229] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:55:13.486370 5767 raft_consensus.cc:2258] T b795eee02f1942c19f5e2d9379cce1a0 P e6e8d4ed07ed4bd19b7eb3abb7235b6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:55:13.486518 5768 raft_consensus.cc:2258] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:55:13.487803 5769 tablet_replica.cc:330] T b795eee02f1942c19f5e2d9379cce1a0 P 7f1672c77bc8423cb16c264dff33d153: stopping tablet replica
I20260619 02:55:13.488212 5769 raft_consensus.cc:2229] T b795eee02f1942c19f5e2d9379cce1a0 P 7f1672c77bc8423cb16c264dff33d153 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:55:13.488729 5769 raft_consensus.cc:2258] T b795eee02f1942c19f5e2d9379cce1a0 P 7f1672c77bc8423cb16c264dff33d153 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:55:13.488912 5768 ts_tablet_manager.cc:1905] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:55:13.488946 5767 ts_tablet_manager.cc:1905] T b795eee02f1942c19f5e2d9379cce1a0 P e6e8d4ed07ed4bd19b7eb3abb7235b6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:55:13.490870 5769 ts_tablet_manager.cc:1905] T b795eee02f1942c19f5e2d9379cce1a0 P 7f1672c77bc8423cb16c264dff33d153: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:55:13.495982 5768 ts_tablet_manager.cc:1918] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:55:13.496163 5768 log.cc:1196] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-0/wal/wals/b795eee02f1942c19f5e2d9379cce1a0
I20260619 02:55:13.496459 5767 ts_tablet_manager.cc:1918] T b795eee02f1942c19f5e2d9379cce1a0 P e6e8d4ed07ed4bd19b7eb3abb7235b6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:55:13.496726 5738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:13.473764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44738 (local address 127.18.134.1:36663)
0619 02:55:13.473950 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:13.473973 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:13.488116 (+ 14143us) server_negotiation.cc:407] Connection header received
0619 02:55:13.488410 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:13.488415 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:13.488521 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:13.488621 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:13.489303 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.490414 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.492211 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.492811 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.492927 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:13.494678 (+ 1751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:13.494691 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:13.494710 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:13.495639 (+ 929us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:13.495729 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:13.495757 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:13.495849 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:13.496155 (+ 306us) server_negotiation.cc:299] Negotiation successful
0619 02:55:13.496283 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 02:55:13.496809 5768 ts_tablet_manager.cc:1939] T b795eee02f1942c19f5e2d9379cce1a0 P 37389331753d43bd9d6823d17ebc670a: Deleting consensus metadata
I20260619 02:55:13.497040 5767 log.cc:1196] T b795eee02f1942c19f5e2d9379cce1a0 P e6e8d4ed07ed4bd19b7eb3abb7235b6c: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-1/wal/wals/b795eee02f1942c19f5e2d9379cce1a0
I20260619 02:55:13.497638 5767 ts_tablet_manager.cc:1939] T b795eee02f1942c19f5e2d9379cce1a0 P e6e8d4ed07ed4bd19b7eb3abb7235b6c: Deleting consensus metadata
I20260619 02:55:13.498100 5460 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:13.498767 5265 catalog_manager.cc:4841] TS 37389331753d43bd9d6823d17ebc670a (127.18.134.1:36663): tablet b795eee02f1942c19f5e2d9379cce1a0 (table test-table [id=35ccf4e34b394436bb92af3b861da4ac]) successfully deleted
I20260619 02:55:13.499027 5769 ts_tablet_manager.cc:1918] T b795eee02f1942c19f5e2d9379cce1a0 P 7f1672c77bc8423cb16c264dff33d153: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:55:13.499235 5769 log.cc:1196] T b795eee02f1942c19f5e2d9379cce1a0 P 7f1672c77bc8423cb16c264dff33d153: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781837410515220-18968-0/minicluster-data/ts-2/wal/wals/b795eee02f1942c19f5e2d9379cce1a0
I20260619 02:55:13.499758 5266 catalog_manager.cc:4841] TS e6e8d4ed07ed4bd19b7eb3abb7235b6c (127.18.134.2:46145): tablet b795eee02f1942c19f5e2d9379cce1a0 (table test-table [id=35ccf4e34b394436bb92af3b861da4ac]) successfully deleted
I20260619 02:55:13.499868 5769 ts_tablet_manager.cc:1939] T b795eee02f1942c19f5e2d9379cce1a0 P 7f1672c77bc8423cb16c264dff33d153: Deleting consensus metadata
I20260619 02:55:13.501838 5265 catalog_manager.cc:4841] TS 7f1672c77bc8423cb16c264dff33d153 (127.18.134.3:43741): tablet b795eee02f1942c19f5e2d9379cce1a0 (table test-table [id=35ccf4e34b394436bb92af3b861da4ac]) successfully deleted
I20260619 02:55:13.593079 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 5338
I20260619 02:55:13.607553 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 5468
I20260619 02:55:13.614481 5737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:13.608365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33394 (local address 127.18.134.2:46145)
0619 02:55:13.608555 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:13.608562 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:13.608582 (+ 20us) server_negotiation.cc:407] Connection header received
0619 02:55:13.608678 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:13.608684 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:13.608787 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:13.608874 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:13.609266 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.610023 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.611460 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.611882 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.611993 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:13.613028 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:13.613036 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:13.613050 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:13.613703 (+ 653us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:13.613760 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:13.613783 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:13.613926 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:13.614052 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 02:55:13.614176 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 02:55:13.615207 5590 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:13.702683 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 5468
I20260619 02:55:13.716303 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 5598
I20260619 02:55:13.723399 5742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:13.717108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44736 (local address 127.18.134.3:43741)
0619 02:55:13.717289 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:13.717293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:13.717359 (+ 66us) server_negotiation.cc:407] Connection header received
0619 02:55:13.717528 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:13.717532 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:13.717631 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:13.717711 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:13.718219 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.718988 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.720359 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:13.720766 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:13.720866 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:13.721929 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:13.721938 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:13.721953 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:13.722507 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:13.722574 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:13.722595 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:13.722760 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:13.722920 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 02:55:13.723037 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 02:55:13.724102 5720 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:13.812743 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 5598
I20260619 02:55:13.826555 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 5248
I20260619 02:55:13.827739 5309 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:13.934098 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 5248
2026-06-19T02:55:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2657 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:55:14 dist-test-slave-9mr0 krb5kdc[5777](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:55:14 dist-test-slave-9mr0 krb5kdc[5777](info): set up 2 sockets
Jun 19 02:55:14 dist-test-slave-9mr0 krb5kdc[5777](info): commencing operation
krb5kdc: starting...
W20260619 02:55:14.954909 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.984s user 0.006s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:55:15 dist-test-slave-9mr0 krb5kdc[5777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:55:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:55:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:55:15.206738 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:36013
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:35315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:36013
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:15.523437 5790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:15.523733 5790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:15.523828 5790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:15.531072 5790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:15.531190 5790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:15.531243 5790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:15.531301 5790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:15.531343 5790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:15.540995 5790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:35315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:36013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:36013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:15.542235 5790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:15.543460 5790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:15.555018 5795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:15.555088 5796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:15.555063 5798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:15.555747 5790 server_base.cc:1034] running on GCE node
I20260619 02:55:15.556623 5790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:15.558012 5790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:15.559209 5790 hybrid_clock.cc:648] HybridClock initialized: now 1781837715559146 us; error 44 us; skew 500 ppm
I20260619 02:55:15.559513 5790 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:15 dist-test-slave-9mr0 krb5kdc[5777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837715, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:15.565424 5790 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:55:15.566862 5790 webserver.cc:460] Webserver started at http://127.18.134.62:43025/ using document root <none> and password file <none>
I20260619 02:55:15.567516 5790 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:15.567655 5790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:15.567914 5790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:15.570828 5790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "19de670d70dd4641b965d46c6bd76b76"
format_stamp: "Formatted at 2026-06-19 02:55:15 on dist-test-slave-9mr0"
server_key: "74b0ad7f65984314da5eac909e7bfd92"
server_key_iv: "2b3d2c35c94888cd3c78001b53d5b141"
server_key_version: "encryptionkey@0"
I20260619 02:55:15.571792 5790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "19de670d70dd4641b965d46c6bd76b76"
format_stamp: "Formatted at 2026-06-19 02:55:15 on dist-test-slave-9mr0"
server_key: "74b0ad7f65984314da5eac909e7bfd92"
server_key_iv: "2b3d2c35c94888cd3c78001b53d5b141"
server_key_version: "encryptionkey@0"
I20260619 02:55:15.576606 5790 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 02:55:15.579875 5805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:15.581085 5790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:55:15.581224 5790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "19de670d70dd4641b965d46c6bd76b76"
format_stamp: "Formatted at 2026-06-19 02:55:15 on dist-test-slave-9mr0"
server_key: "74b0ad7f65984314da5eac909e7bfd92"
server_key_iv: "2b3d2c35c94888cd3c78001b53d5b141"
server_key_version: "encryptionkey@0"
I20260619 02:55:15.581374 5790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:15.608723 5790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:15.612418 5790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:15.612658 5790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:15.629957 5790 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:36013
I20260619 02:55:15.629964 5856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:36013 every 8 connection(s)
I20260619 02:55:15.631372 5790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:55:15.635663 5857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:15.638970 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 5790
I20260619 02:55:15.639142 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:55:15.639554 18968 external_mini_cluster.cc:1428] Setting key 5e9a87554fb2693ef07486bab451d7b8
I20260619 02:55:15.647740 5857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19de670d70dd4641b965d46c6bd76b76: Bootstrap starting.
I20260619 02:55:15.651443 5857 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19de670d70dd4641b965d46c6bd76b76: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:15.652727 5857 log.cc:824] T 00000000000000000000000000000000 P 19de670d70dd4641b965d46c6bd76b76: Log is configured to *not* fsync() on all Append() calls
Jun 19 02:55:15 dist-test-slave-9mr0 krb5kdc[5777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:55:15.655915 5857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19de670d70dd4641b965d46c6bd76b76: No bootstrap required, opened a new log
I20260619 02:55:15.663559 5860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:15.641389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44410 (local address 127.18.134.62:36013)
0619 02:55:15.642113 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:15.642124 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:15.642164 (+ 40us) server_negotiation.cc:407] Connection header received
0619 02:55:15.643806 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:15.643829 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:15.644249 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:15.644582 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:15.646269 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:15.647608 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:15.649359 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:15.650020 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:15.650141 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:15.655057 (+ 4916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:15.655069 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:15.655089 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:15.655140 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:15.658990 (+ 3850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:15.659651 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:15.659663 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:15.659672 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:15.659878 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:15.660262 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:15.660267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:15.660270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:15.660958 (+ 688us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:15.661251 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:15.661634 (+ 383us) server_negotiation.cc:299] Negotiation successful
0619 02:55:15.662081 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":332,"threads_started":1}
I20260619 02:55:15.666071 5857 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19de670d70dd4641b965d46c6bd76b76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19de670d70dd4641b965d46c6bd76b76" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 36013 } }
I20260619 02:55:15.666435 5857 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19de670d70dd4641b965d46c6bd76b76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:15.666543 5857 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19de670d70dd4641b965d46c6bd76b76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19de670d70dd4641b965d46c6bd76b76, State: Initialized, Role: FOLLOWER
I20260619 02:55:15.667114 5857 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19de670d70dd4641b965d46c6bd76b76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19de670d70dd4641b965d46c6bd76b76" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 36013 } }
I20260619 02:55:15.667325 5857 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19de670d70dd4641b965d46c6bd76b76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:55:15.667443 5857 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19de670d70dd4641b965d46c6bd76b76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:55:15.667591 5857 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19de670d70dd4641b965d46c6bd76b76 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:15.669883 5857 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19de670d70dd4641b965d46c6bd76b76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19de670d70dd4641b965d46c6bd76b76" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 36013 } }
I20260619 02:55:15.670351 5857 leader_election.cc:302] T 00000000000000000000000000000000 P 19de670d70dd4641b965d46c6bd76b76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19de670d70dd4641b965d46c6bd76b76; no voters:
I20260619 02:55:15.670898 5857 leader_election.cc:288] T 00000000000000000000000000000000 P 19de670d70dd4641b965d46c6bd76b76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:55:15.671077 5862 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19de670d70dd4641b965d46c6bd76b76 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:15.671698 5862 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19de670d70dd4641b965d46c6bd76b76 [term 1 LEADER]: Becoming Leader. State: Replica: 19de670d70dd4641b965d46c6bd76b76, State: Running, Role: LEADER
I20260619 02:55:15.672232 5862 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19de670d70dd4641b965d46c6bd76b76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19de670d70dd4641b965d46c6bd76b76" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 36013 } }
I20260619 02:55:15.672834 5857 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19de670d70dd4641b965d46c6bd76b76 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:55:15.675122 5863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19de670d70dd4641b965d46c6bd76b76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19de670d70dd4641b965d46c6bd76b76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19de670d70dd4641b965d46c6bd76b76" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 36013 } } }
I20260619 02:55:15.675149 5864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19de670d70dd4641b965d46c6bd76b76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19de670d70dd4641b965d46c6bd76b76. Latest consensus state: current_term: 1 leader_uuid: "19de670d70dd4641b965d46c6bd76b76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19de670d70dd4641b965d46c6bd76b76" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 36013 } } }
I20260619 02:55:15.675570 5863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19de670d70dd4641b965d46c6bd76b76 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:15.675570 5864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19de670d70dd4641b965d46c6bd76b76 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:15.677273 5867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:55:15.681885 5867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:55:15.690371 5867 catalog_manager.cc:1347] Generated new cluster ID: 9d2c512db0eb4957a8debfccdf0440c5
I20260619 02:55:15.690495 5867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:55:15.708525 5867 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:55:15.709820 5867 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:55:15.716264 5867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19de670d70dd4641b965d46c6bd76b76: Generated new TSK 0
I20260619 02:55:15.717052 5867 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.134.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
Entry for principal kudu/127.18.134.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab.
I20260619 02:55:15.771482 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:0
--local_ip_for_outbound_sockets=127.18.134.1
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:36013
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:35315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:16.087848 5883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:16.088119 5883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:16.088236 5883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:16.095520 5883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:16.095634 5883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:16.095803 5883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:55:16.105443 5883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:35315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.1.keytab
--principal=kudu/127.18.134.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:36013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:16.106695 5883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:16.108043 5883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:16.120219 5888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:16.120280 5889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:16.120250 5891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:16.121443 5883 server_base.cc:1034] running on GCE node
I20260619 02:55:16.122040 5883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:16.123044 5883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:16.124235 5883 hybrid_clock.cc:648] HybridClock initialized: now 1781837716124186 us; error 39 us; skew 500 ppm
I20260619 02:55:16.124528 5883 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:16 dist-test-slave-9mr0 krb5kdc[5777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837716, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:16.127547 5883 init.cc:377] Logged in from keytab as kudu/127.18.134.1@KRBTEST.COM (short username kudu)
I20260619 02:55:16.128991 5883 webserver.cc:460] Webserver started at http://127.18.134.1:46349/ using document root <none> and password file <none>
I20260619 02:55:16.129647 5883 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:16.129743 5883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:16.129994 5883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:16.132936 5883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-0/data/instance:
uuid: "d76e80ef0ea24f8caec67b59742cac9f"
format_stamp: "Formatted at 2026-06-19 02:55:16 on dist-test-slave-9mr0"
server_key: "7f0436cd8dfdd12dcfeabba79febe572"
server_key_iv: "ae463abe20e8747ee912b9b19314db23"
server_key_version: "encryptionkey@0"
I20260619 02:55:16.133647 5883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance:
uuid: "d76e80ef0ea24f8caec67b59742cac9f"
format_stamp: "Formatted at 2026-06-19 02:55:16 on dist-test-slave-9mr0"
server_key: "7f0436cd8dfdd12dcfeabba79febe572"
server_key_iv: "ae463abe20e8747ee912b9b19314db23"
server_key_version: "encryptionkey@0"
I20260619 02:55:16.138792 5883 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260619 02:55:16.142184 5898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:16.143543 5883 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:55:16.143707 5883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "d76e80ef0ea24f8caec67b59742cac9f"
format_stamp: "Formatted at 2026-06-19 02:55:16 on dist-test-slave-9mr0"
server_key: "7f0436cd8dfdd12dcfeabba79febe572"
server_key_iv: "ae463abe20e8747ee912b9b19314db23"
server_key_version: "encryptionkey@0"
I20260619 02:55:16.143886 5883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:16.159116 5883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:16.163192 5883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:16.163463 5883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:16.164362 5883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:55:16.165639 5883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:55:16.165714 5883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:16.165834 5883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:55:16.165886 5883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:16.196076 5883 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:37825
I20260619 02:55:16.196125 6010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:37825 every 8 connection(s)
I20260619 02:55:16.197307 5883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:55:16.204165 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 5883
I20260619 02:55:16.204320 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance
I20260619 02:55:16.204731 18968 external_mini_cluster.cc:1428] Setting key 552e1ce7a7d7fb07e5c0918db5c1cf58
Jun 19 02:55:16 dist-test-slave-9mr0 krb5kdc[5777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837716, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.1@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:55:16.216507 6013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:16.199777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:33141 (local address 127.18.134.62:36013)
0619 02:55:16.200202 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:16.200207 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:16.201113 (+ 906us) server_negotiation.cc:407] Connection header received
0619 02:55:16.202913 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:16.202918 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:16.202999 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:16.203092 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:16.204624 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:16.205390 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:16.207231 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:16.207617 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:16.207732 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:16.210972 (+ 3240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:16.210976 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:16.210980 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:16.211012 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:16.213391 (+ 2379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:16.214068 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:16.214072 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:16.214074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:16.214195 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:16.214590 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:16.214594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:16.214596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:16.214904 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:16.215035 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:16.215785 (+ 750us) server_negotiation.cc:299] Negotiation successful
0619 02:55:16.216032 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":210,"threads_started":1}
I20260619 02:55:16.217375 6014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:16.200275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:36013 (local address 127.18.134.1:33141)
0619 02:55:16.200953 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:16.201000 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:16.202654 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:16.203340 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:16.203348 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:16.203855 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:16.204465 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:16.204484 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:16.205542 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:16.205548 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:16.207093 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:16.207105 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:16.207833 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:16.207840 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:16.208064 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:16.208675 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:55:16.208697 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:16.210770 (+ 2073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:16.213522 (+ 2752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:16.213531 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:16.213549 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:16.213950 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:16.214292 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:16.214297 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:16.214301 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:16.214480 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:16.215025 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:16.215033 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:16.215423 (+ 390us) client_negotiation.cc:769] Sending connection context
0619 02:55:16.215663 (+ 240us) client_negotiation.cc:240] Negotiation successful
0619 02:55:16.216208 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":266,"threads_started":1}
I20260619 02:55:16.218914 6011 heartbeater.cc:347] Connected to a master server at 127.18.134.62:36013
I20260619 02:55:16.219246 6011 heartbeater.cc:464] Registering TS with master...
I20260619 02:55:16.220083 6011 heartbeater.cc:511] Master 127.18.134.62:36013 requested a full tablet report, sending...
I20260619 02:55:16.222330 5822 ts_manager.cc:194] Registered new tserver with Master: d76e80ef0ea24f8caec67b59742cac9f (127.18.134.1:37825)
I20260619 02:55:16.224331 5822 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.1@KRBTEST.COM'} at 127.18.134.1:33141
WARNING: no policy specified for kudu/127.18.134.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
Entry for principal kudu/127.18.134.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab.
I20260619 02:55:16.258948 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:0
--local_ip_for_outbound_sockets=127.18.134.2
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:36013
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:35315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:16.565187 6017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:16.565447 6017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:16.565550 6017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:16.572700 6017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:16.572808 6017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:16.572925 6017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:55:16.582791 6017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:35315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.2.keytab
--principal=kudu/127.18.134.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:36013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:16.584023 6017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:16.585280 6017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:16.597791 6023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:16.597863 6025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:16.597831 6022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:16.598860 6017 server_base.cc:1034] running on GCE node
I20260619 02:55:16.599447 6017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:16.600433 6017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:16.601625 6017 hybrid_clock.cc:648] HybridClock initialized: now 1781837716601570 us; error 48 us; skew 500 ppm
I20260619 02:55:16.601907 6017 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:16 dist-test-slave-9mr0 krb5kdc[5777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837716, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:16.604842 6017 init.cc:377] Logged in from keytab as kudu/127.18.134.2@KRBTEST.COM (short username kudu)
I20260619 02:55:16.606251 6017 webserver.cc:460] Webserver started at http://127.18.134.2:39025/ using document root <none> and password file <none>
I20260619 02:55:16.606946 6017 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:16.607066 6017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:16.607352 6017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:16.610183 6017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-1/data/instance:
uuid: "74387e86771e4d61a531d606f5ad3d28"
format_stamp: "Formatted at 2026-06-19 02:55:16 on dist-test-slave-9mr0"
server_key: "959b552fe668814420a3320619f550a7"
server_key_iv: "b272dd0bee4a63a43c1d5ca82d35ca94"
server_key_version: "encryptionkey@0"
I20260619 02:55:16.610921 6017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance:
uuid: "74387e86771e4d61a531d606f5ad3d28"
format_stamp: "Formatted at 2026-06-19 02:55:16 on dist-test-slave-9mr0"
server_key: "959b552fe668814420a3320619f550a7"
server_key_iv: "b272dd0bee4a63a43c1d5ca82d35ca94"
server_key_version: "encryptionkey@0"
I20260619 02:55:16.615811 6017 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 02:55:16.619460 6032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:16.620846 6017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:55:16.621049 6017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "74387e86771e4d61a531d606f5ad3d28"
format_stamp: "Formatted at 2026-06-19 02:55:16 on dist-test-slave-9mr0"
server_key: "959b552fe668814420a3320619f550a7"
server_key_iv: "b272dd0bee4a63a43c1d5ca82d35ca94"
server_key_version: "encryptionkey@0"
I20260619 02:55:16.621259 6017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:16.655018 6017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:16.659101 6017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:16.659343 6017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:16.660282 6017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:55:16.661564 6017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:55:16.661635 6017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:16.661732 6017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:55:16.661775 6017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:16.690249 6017 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:36717
I20260619 02:55:16.690320 6144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:36717 every 8 connection(s)
I20260619 02:55:16.691565 6017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:55:16.700591 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 6017
I20260619 02:55:16.700767 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance
I20260619 02:55:16.701278 18968 external_mini_cluster.cc:1428] Setting key bfb17f05cc42ab6e0a89182c33df7a8d
Jun 19 02:55:16 dist-test-slave-9mr0 krb5kdc[5777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837716, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.2@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:55:16.711252 6013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:16.693976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:58743 (local address 127.18.134.62:36013)
0619 02:55:16.694145 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:16.694150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:16.695305 (+ 1155us) server_negotiation.cc:407] Connection header received
0619 02:55:16.697072 (+ 1767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:16.697077 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:16.697158 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:16.697245 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:16.698693 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:16.699403 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:16.700912 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:16.701526 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:16.701654 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:16.705408 (+ 3754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:16.705413 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:16.705416 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:16.705448 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:16.707873 (+ 2425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:16.708686 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:16.708691 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:16.708694 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:16.708856 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:16.709417 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:16.709421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:16.709423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:16.709710 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:16.709830 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:16.710613 (+ 783us) server_negotiation.cc:299] Negotiation successful
0619 02:55:16.710912 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 02:55:16.712080 6147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:16.694443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:36013 (local address 127.18.134.2:58743)
0619 02:55:16.695154 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:16.695193 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:16.696815 (+ 1622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:16.697477 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:16.697485 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:16.697977 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:16.698510 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:16.698529 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:16.699546 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:16.699550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:16.700777 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:16.700792 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:16.701728 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:16.701733 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:16.701969 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:16.702610 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:55:16.702670 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:16.705187 (+ 2517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:16.707998 (+ 2811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:16.708008 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:16.708025 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:16.708550 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:16.709072 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:16.709076 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:16.709079 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:16.709306 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:16.709823 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:16.709830 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:16.710246 (+ 416us) client_negotiation.cc:769] Sending connection context
0619 02:55:16.710539 (+ 293us) client_negotiation.cc:240] Negotiation successful
0619 02:55:16.711015 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":308,"threads_started":1}
I20260619 02:55:16.713406 6145 heartbeater.cc:347] Connected to a master server at 127.18.134.62:36013
I20260619 02:55:16.713732 6145 heartbeater.cc:464] Registering TS with master...
I20260619 02:55:16.714537 6145 heartbeater.cc:511] Master 127.18.134.62:36013 requested a full tablet report, sending...
I20260619 02:55:16.716034 5822 ts_manager.cc:194] Registered new tserver with Master: 74387e86771e4d61a531d606f5ad3d28 (127.18.134.2:36717)
I20260619 02:55:16.716854 5822 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.2@KRBTEST.COM'} at 127.18.134.2:58743
WARNING: no policy specified for kudu/127.18.134.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
Entry for principal kudu/127.18.134.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab.
I20260619 02:55:16.753863 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:0
--local_ip_for_outbound_sockets=127.18.134.3
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:36013
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:35315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:17.073547 6150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:17.073982 6150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:17.074157 6150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:17.081818 6150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:17.081928 6150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:17.082116 6150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:55:17.092163 6150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:35315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.3.keytab
--principal=kudu/127.18.134.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:36013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:17.093432 6150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:17.094781 6150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:17.107234 6156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:17.107586 6155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:17.107733 6158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:17.108538 6150 server_base.cc:1034] running on GCE node
I20260619 02:55:17.109136 6150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:17.110066 6150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:17.111261 6150 hybrid_clock.cc:648] HybridClock initialized: now 1781837717111212 us; error 44 us; skew 500 ppm
I20260619 02:55:17.111568 6150 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:17 dist-test-slave-9mr0 krb5kdc[5777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837717, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:17.114748 6150 init.cc:377] Logged in from keytab as kudu/127.18.134.3@KRBTEST.COM (short username kudu)
I20260619 02:55:17.116240 6150 webserver.cc:460] Webserver started at http://127.18.134.3:39567/ using document root <none> and password file <none>
I20260619 02:55:17.116940 6150 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:17.117074 6150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:17.117441 6150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:17.120375 6150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-2/data/instance:
uuid: "65ca759dd2b04dbbbdeea2f481cc29e1"
format_stamp: "Formatted at 2026-06-19 02:55:17 on dist-test-slave-9mr0"
server_key: "0281ec1a9e2d970b2f0aa4450b866e3e"
server_key_iv: "990fd892787840aef880f8e548fac446"
server_key_version: "encryptionkey@0"
I20260619 02:55:17.121096 6150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance:
uuid: "65ca759dd2b04dbbbdeea2f481cc29e1"
format_stamp: "Formatted at 2026-06-19 02:55:17 on dist-test-slave-9mr0"
server_key: "0281ec1a9e2d970b2f0aa4450b866e3e"
server_key_iv: "990fd892787840aef880f8e548fac446"
server_key_version: "encryptionkey@0"
I20260619 02:55:17.126340 6150 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 02:55:17.129765 6165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:17.131186 6150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:55:17.131345 6150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "65ca759dd2b04dbbbdeea2f481cc29e1"
format_stamp: "Formatted at 2026-06-19 02:55:17 on dist-test-slave-9mr0"
server_key: "0281ec1a9e2d970b2f0aa4450b866e3e"
server_key_iv: "990fd892787840aef880f8e548fac446"
server_key_version: "encryptionkey@0"
I20260619 02:55:17.131531 6150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:17.148645 6150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:17.152508 6150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:17.152761 6150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:17.153723 6150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:55:17.155014 6150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:55:17.155090 6150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:17.155226 6150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:55:17.155297 6150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:17.186867 6150 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:43273
I20260619 02:55:17.186905 6277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:43273 every 8 connection(s)
I20260619 02:55:17.188056 6150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:55:17.196298 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 6150
I20260619 02:55:17.196501 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance
I20260619 02:55:17.197129 18968 external_mini_cluster.cc:1428] Setting key 28abc630b407bd2105208e6f21ac4414
Jun 19 02:55:17 dist-test-slave-9mr0 krb5kdc[5777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837717, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.3@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:55:17.210263 6013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:17.190703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:45773 (local address 127.18.134.62:36013)
0619 02:55:17.191007 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:17.191013 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:17.192250 (+ 1237us) server_negotiation.cc:407] Connection header received
0619 02:55:17.194823 (+ 2573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:17.194827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:17.194902 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:17.195029 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:17.197170 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.198336 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.201340 (+ 3004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.201772 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.201867 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:17.205174 (+ 3307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:17.205179 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:17.205183 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:17.205223 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:17.207637 (+ 2414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:17.208338 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:17.208343 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:17.208345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:17.208464 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:17.208811 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:17.208814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:17.208816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:17.209126 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:17.209232 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:17.209808 (+ 576us) server_negotiation.cc:299] Negotiation successful
0619 02:55:17.210021 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260619 02:55:17.211195 6280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:17.191069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:36013 (local address 127.18.134.3:45773)
0619 02:55:17.192054 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:17.192129 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:17.194456 (+ 2327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:17.195322 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:17.195334 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:17.196068 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:17.196936 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:17.196962 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.198846 (+ 1884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.198852 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:17.201210 (+ 2358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:17.201223 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.201940 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.201945 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:17.202189 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:17.202868 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:55:17.202890 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:17.204962 (+ 2072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:17.207758 (+ 2796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:17.207765 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:17.207777 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:17.208215 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:17.208546 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:17.208549 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:17.208552 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:17.208726 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:17.209219 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:17.209224 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:17.209525 (+ 301us) client_negotiation.cc:769] Sending connection context
0619 02:55:17.209737 (+ 212us) client_negotiation.cc:240] Negotiation successful
0619 02:55:17.210137 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":321,"threads_started":1}
I20260619 02:55:17.212633 6278 heartbeater.cc:347] Connected to a master server at 127.18.134.62:36013
I20260619 02:55:17.213023 6278 heartbeater.cc:464] Registering TS with master...
I20260619 02:55:17.213783 6278 heartbeater.cc:511] Master 127.18.134.62:36013 requested a full tablet report, sending...
I20260619 02:55:17.215144 5821 ts_manager.cc:194] Registered new tserver with Master: 65ca759dd2b04dbbbdeea2f481cc29e1 (127.18.134.3:43273)
I20260619 02:55:17.216035 5821 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.134.3@KRBTEST.COM'} at 127.18.134.3:45773
I20260619 02:55:17.225641 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:55:17.228137 6011 heartbeater.cc:503] Master 127.18.134.62:36013 was elected leader, sending a full tablet report...
I20260619 02:55:17.242158 6013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:17.229871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44426 (local address 127.18.134.62:36013)
0619 02:55:17.230042 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:17.230047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:17.230481 (+ 434us) server_negotiation.cc:407] Connection header received
0619 02:55:17.230658 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:17.230663 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:17.230734 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:17.230813 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:17.232150 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.232851 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.234170 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.234656 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.234781 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:17.237149 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:17.237154 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:17.237158 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:17.237190 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:17.239638 (+ 2448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:17.240297 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:17.240301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:17.240303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:17.240429 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:17.240829 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:17.240831 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:17.240833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:17.241155 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:17.241328 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:17.241746 (+ 418us) server_negotiation.cc:299] Negotiation successful
0619 02:55:17.241920 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 02:55:17.246055 5821 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44426:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:55:17.252820 5821 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:55:17.274555 6294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:17.266213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:36717 (local address 127.0.0.1:60506)
0619 02:55:17.266755 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:17.266773 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:17.266889 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:17.267783 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:17.267790 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:17.267817 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:17.268169 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:17.268184 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.269994 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.270000 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:17.272143 (+ 2143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:17.272157 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.273642 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.273648 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:17.273835 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:17.273891 (+ 56us) client_negotiation.cc:769] Sending connection context
0619 02:55:17.274015 (+ 124us) client_negotiation.cc:240] Negotiation successful
0619 02:55:17.274217 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":299,"threads_started":1}
I20260619 02:55:17.274583 6295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:17.266563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60506 (local address 127.18.134.2:36717)
0619 02:55:17.267331 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:17.267338 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:17.267358 (+ 20us) server_negotiation.cc:407] Connection header received
0619 02:55:17.267429 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:17.267434 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:17.267592 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:17.267753 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:17.268358 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.269869 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.272325 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.273454 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.273592 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:17.273809 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:17.274083 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 02:55:17.274255 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":482,"threads_started":1}
I20260619 02:55:17.276113 6292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:17.265274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:37825 (local address 127.0.0.1:46066)
0619 02:55:17.266043 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:17.266058 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:17.266165 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:17.268362 (+ 2197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:17.268366 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:17.268393 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:17.268810 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:17.268824 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.270713 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.270718 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:17.273129 (+ 2411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:17.273144 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.275246 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.275252 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:17.275429 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:17.275461 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 02:55:17.275539 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 02:55:17.275667 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"mutex_wait_us":99,"thread_start_us":740,"threads_started":1}
I20260619 02:55:17.276283 6290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:17.264790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:43273 (local address 127.0.0.1:55636)
0619 02:55:17.265596 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:17.265636 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:17.265845 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:17.268938 (+ 3093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:17.268942 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:17.268965 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:17.269308 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:17.269320 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.271297 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.271301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:17.273347 (+ 2046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:17.273359 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.275355 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.275361 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:17.275520 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:17.275551 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 02:55:17.275627 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 02:55:17.275751 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"mutex_wait_us":1,"thread_start_us":599,"threads_started":1}
I20260619 02:55:17.276779 6293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:17.265592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46066 (local address 127.18.134.1:37825)
0619 02:55:17.267127 (+ 1535us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:17.267137 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:17.267169 (+ 32us) server_negotiation.cc:407] Connection header received
0619 02:55:17.267274 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:17.267283 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:17.267558 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:17.267757 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:17.269008 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.270554 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.273332 (+ 2778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.275062 (+ 1730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.275187 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:17.275407 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:17.275598 (+ 191us) server_negotiation.cc:299] Negotiation successful
0619 02:55:17.275756 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1365,"thread_start_us":1222,"threads_started":1}
I20260619 02:55:17.277421 6291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:17.264889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55636 (local address 127.18.134.3:43273)
0619 02:55:17.268382 (+ 3493us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:17.268390 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:17.268433 (+ 43us) server_negotiation.cc:407] Connection header received
0619 02:55:17.268536 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:17.268542 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:17.268742 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:17.269512 (+ 770us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:17.269873 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.271156 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.273512 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.275126 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.276555 (+ 1429us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:17.276804 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:17.276971 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 02:55:17.277122 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3354,"thread_start_us":3219,"threads_started":1}
I20260619 02:55:17.278577 6080 tablet_service.cc:1467] Processing CreateTablet for tablet 6c64880460744750b976552cb8803174 (DEFAULT_TABLE table=test-table [id=16a5bddbcc054673bcd9d907b8542c33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:55:17.280164 6080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c64880460744750b976552cb8803174. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:17.281191 6213 tablet_service.cc:1467] Processing CreateTablet for tablet 6c64880460744750b976552cb8803174 (DEFAULT_TABLE table=test-table [id=16a5bddbcc054673bcd9d907b8542c33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:55:17.281661 5946 tablet_service.cc:1467] Processing CreateTablet for tablet 6c64880460744750b976552cb8803174 (DEFAULT_TABLE table=test-table [id=16a5bddbcc054673bcd9d907b8542c33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:55:17.282593 6213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c64880460744750b976552cb8803174. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:17.282816 5946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c64880460744750b976552cb8803174. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:17.293183 6296 tablet_bootstrap.cc:492] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f: Bootstrap starting.
I20260619 02:55:17.296507 6298 tablet_bootstrap.cc:492] T 6c64880460744750b976552cb8803174 P 65ca759dd2b04dbbbdeea2f481cc29e1: Bootstrap starting.
I20260619 02:55:17.297467 6296 tablet_bootstrap.cc:654] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:17.298744 6296 log.cc:824] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:17.299372 6297 tablet_bootstrap.cc:492] T 6c64880460744750b976552cb8803174 P 74387e86771e4d61a531d606f5ad3d28: Bootstrap starting.
I20260619 02:55:17.299999 6298 tablet_bootstrap.cc:654] T 6c64880460744750b976552cb8803174 P 65ca759dd2b04dbbbdeea2f481cc29e1: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:17.300998 6298 log.cc:824] T 6c64880460744750b976552cb8803174 P 65ca759dd2b04dbbbdeea2f481cc29e1: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:17.302663 6297 tablet_bootstrap.cc:654] T 6c64880460744750b976552cb8803174 P 74387e86771e4d61a531d606f5ad3d28: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:17.302760 6296 tablet_bootstrap.cc:492] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f: No bootstrap required, opened a new log
I20260619 02:55:17.303058 6296 ts_tablet_manager.cc:1397] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260619 02:55:17.303790 6297 log.cc:824] T 6c64880460744750b976552cb8803174 P 74387e86771e4d61a531d606f5ad3d28: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:17.306138 6297 tablet_bootstrap.cc:492] T 6c64880460744750b976552cb8803174 P 74387e86771e4d61a531d606f5ad3d28: No bootstrap required, opened a new log
I20260619 02:55:17.306376 6297 ts_tablet_manager.cc:1397] T 6c64880460744750b976552cb8803174 P 74387e86771e4d61a531d606f5ad3d28: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260619 02:55:17.309118 6298 tablet_bootstrap.cc:492] T 6c64880460744750b976552cb8803174 P 65ca759dd2b04dbbbdeea2f481cc29e1: No bootstrap required, opened a new log
I20260619 02:55:17.309371 6298 ts_tablet_manager.cc:1397] T 6c64880460744750b976552cb8803174 P 65ca759dd2b04dbbbdeea2f481cc29e1: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260619 02:55:17.313223 6297 raft_consensus.cc:348] T 6c64880460744750b976552cb8803174 P 74387e86771e4d61a531d606f5ad3d28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca759dd2b04dbbbdeea2f481cc29e1" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43273 } } peers { permanent_uuid: "74387e86771e4d61a531d606f5ad3d28" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 36717 } } peers { permanent_uuid: "d76e80ef0ea24f8caec67b59742cac9f" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 37825 } }
I20260619 02:55:17.313123 6296 raft_consensus.cc:348] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca759dd2b04dbbbdeea2f481cc29e1" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43273 } } peers { permanent_uuid: "74387e86771e4d61a531d606f5ad3d28" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 36717 } } peers { permanent_uuid: "d76e80ef0ea24f8caec67b59742cac9f" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 37825 } }
I20260619 02:55:17.313544 6296 raft_consensus.cc:374] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:17.313549 6297 raft_consensus.cc:374] T 6c64880460744750b976552cb8803174 P 74387e86771e4d61a531d606f5ad3d28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:17.313637 6296 raft_consensus.cc:729] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d76e80ef0ea24f8caec67b59742cac9f, State: Initialized, Role: FOLLOWER
I20260619 02:55:17.313642 6297 raft_consensus.cc:729] T 6c64880460744750b976552cb8803174 P 74387e86771e4d61a531d606f5ad3d28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74387e86771e4d61a531d606f5ad3d28, State: Initialized, Role: FOLLOWER
I20260619 02:55:17.314312 6297 consensus_queue.cc:260] T 6c64880460744750b976552cb8803174 P 74387e86771e4d61a531d606f5ad3d28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca759dd2b04dbbbdeea2f481cc29e1" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43273 } } peers { permanent_uuid: "74387e86771e4d61a531d606f5ad3d28" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 36717 } } peers { permanent_uuid: "d76e80ef0ea24f8caec67b59742cac9f" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 37825 } }
I20260619 02:55:17.314316 6296 consensus_queue.cc:260] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca759dd2b04dbbbdeea2f481cc29e1" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43273 } } peers { permanent_uuid: "74387e86771e4d61a531d606f5ad3d28" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 36717 } } peers { permanent_uuid: "d76e80ef0ea24f8caec67b59742cac9f" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 37825 } }
I20260619 02:55:17.316329 6145 heartbeater.cc:503] Master 127.18.134.62:36013 was elected leader, sending a full tablet report...
I20260619 02:55:17.316900 6297 ts_tablet_manager.cc:1428] T 6c64880460744750b976552cb8803174 P 74387e86771e4d61a531d606f5ad3d28: Time spent starting tablet: real 0.010s user 0.007s sys 0.004s
I20260619 02:55:17.318030 6296 ts_tablet_manager.cc:1428] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260619 02:55:17.320520 6298 raft_consensus.cc:348] T 6c64880460744750b976552cb8803174 P 65ca759dd2b04dbbbdeea2f481cc29e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca759dd2b04dbbbdeea2f481cc29e1" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43273 } } peers { permanent_uuid: "74387e86771e4d61a531d606f5ad3d28" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 36717 } } peers { permanent_uuid: "d76e80ef0ea24f8caec67b59742cac9f" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 37825 } }
I20260619 02:55:17.320956 6298 raft_consensus.cc:374] T 6c64880460744750b976552cb8803174 P 65ca759dd2b04dbbbdeea2f481cc29e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:17.321063 6298 raft_consensus.cc:729] T 6c64880460744750b976552cb8803174 P 65ca759dd2b04dbbbdeea2f481cc29e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65ca759dd2b04dbbbdeea2f481cc29e1, State: Initialized, Role: FOLLOWER
I20260619 02:55:17.321666 6298 consensus_queue.cc:260] T 6c64880460744750b976552cb8803174 P 65ca759dd2b04dbbbdeea2f481cc29e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca759dd2b04dbbbdeea2f481cc29e1" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43273 } } peers { permanent_uuid: "74387e86771e4d61a531d606f5ad3d28" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 36717 } } peers { permanent_uuid: "d76e80ef0ea24f8caec67b59742cac9f" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 37825 } }
I20260619 02:55:17.322664 6278 heartbeater.cc:503] Master 127.18.134.62:36013 was elected leader, sending a full tablet report...
I20260619 02:55:17.323118 6298 ts_tablet_manager.cc:1428] T 6c64880460744750b976552cb8803174 P 65ca759dd2b04dbbbdeea2f481cc29e1: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260619 02:55:17.421751 6303 raft_consensus.cc:482] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:55:17.422076 6303 raft_consensus.cc:504] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca759dd2b04dbbbdeea2f481cc29e1" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43273 } } peers { permanent_uuid: "74387e86771e4d61a531d606f5ad3d28" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 36717 } } peers { permanent_uuid: "d76e80ef0ea24f8caec67b59742cac9f" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 37825 } }
I20260619 02:55:17.423728 6303 leader_election.cc:288] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 65ca759dd2b04dbbbdeea2f481cc29e1 (127.18.134.3:43273), 74387e86771e4d61a531d606f5ad3d28 (127.18.134.2:36717)
I20260619 02:55:17.430342 6291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:17.424189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:54229 (local address 127.18.134.3:43273)
0619 02:55:17.424368 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:17.424373 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:17.424924 (+ 551us) server_negotiation.cc:407] Connection header received
0619 02:55:17.425176 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:17.425184 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:17.425287 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:17.425349 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:17.425999 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.426820 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.428731 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.429498 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.429639 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:17.429716 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:17.429983 (+ 267us) server_negotiation.cc:299] Negotiation successful
0619 02:55:17.430108 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 02:55:17.430572 6295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:17.425107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:39955 (local address 127.18.134.2:36717)
0619 02:55:17.425319 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:17.425325 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:17.425636 (+ 311us) server_negotiation.cc:407] Connection header received
0619 02:55:17.425824 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:17.425831 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:17.425891 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:17.425962 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:17.426482 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.427229 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.428973 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.429733 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.429881 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:17.429953 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:17.430205 (+ 252us) server_negotiation.cc:299] Negotiation successful
0619 02:55:17.430328 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 02:55:17.430678 6306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:17.424978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:36717 (local address 127.18.134.1:39955)
0619 02:55:17.425536 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:17.425549 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:17.425697 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:17.425988 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:17.425999 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:17.426022 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:17.426346 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:17.426356 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.427376 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.427381 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:17.428827 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:17.428839 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.429933 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.429939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:17.430073 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:17.430094 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 02:55:17.430169 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 02:55:17.430288 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":24,"thread_start_us":259,"threads_started":1}
I20260619 02:55:17.430728 6305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:17.424159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:43273 (local address 127.18.134.1:54229)
0619 02:55:17.424829 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:17.424847 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:17.424982 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:17.425554 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:17.425557 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:17.425580 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:17.425856 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:17.425864 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.426935 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.426940 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:17.428590 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:17.428605 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.429689 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.429695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:17.429833 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:17.429855 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 02:55:17.429946 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 02:55:17.430071 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"mutex_wait_us":18,"thread_start_us":362,"threads_started":1}
I20260619 02:55:17.431715 6100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c64880460744750b976552cb8803174" candidate_uuid: "d76e80ef0ea24f8caec67b59742cac9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74387e86771e4d61a531d606f5ad3d28" is_pre_election: true
I20260619 02:55:17.431713 6233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c64880460744750b976552cb8803174" candidate_uuid: "d76e80ef0ea24f8caec67b59742cac9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65ca759dd2b04dbbbdeea2f481cc29e1" is_pre_election: true
I20260619 02:55:17.432148 6233 raft_consensus.cc:2454] T 6c64880460744750b976552cb8803174 P 65ca759dd2b04dbbbdeea2f481cc29e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d76e80ef0ea24f8caec67b59742cac9f in term 0.
I20260619 02:55:17.432149 6100 raft_consensus.cc:2454] T 6c64880460744750b976552cb8803174 P 74387e86771e4d61a531d606f5ad3d28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d76e80ef0ea24f8caec67b59742cac9f in term 0.
I20260619 02:55:17.432718 5902 leader_election.cc:302] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74387e86771e4d61a531d606f5ad3d28, d76e80ef0ea24f8caec67b59742cac9f; no voters:
I20260619 02:55:17.433074 6303 raft_consensus.cc:2781] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:55:17.433229 6303 raft_consensus.cc:482] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:55:17.433334 6303 raft_consensus.cc:3037] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:17.435551 6303 raft_consensus.cc:504] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca759dd2b04dbbbdeea2f481cc29e1" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43273 } } peers { permanent_uuid: "74387e86771e4d61a531d606f5ad3d28" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 36717 } } peers { permanent_uuid: "d76e80ef0ea24f8caec67b59742cac9f" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 37825 } }
I20260619 02:55:17.436142 6303 leader_election.cc:288] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f [CANDIDATE]: Term 1 election: Requested vote from peers 65ca759dd2b04dbbbdeea2f481cc29e1 (127.18.134.3:43273), 74387e86771e4d61a531d606f5ad3d28 (127.18.134.2:36717)
I20260619 02:55:17.436615 6233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c64880460744750b976552cb8803174" candidate_uuid: "d76e80ef0ea24f8caec67b59742cac9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65ca759dd2b04dbbbdeea2f481cc29e1"
I20260619 02:55:17.436695 6100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c64880460744750b976552cb8803174" candidate_uuid: "d76e80ef0ea24f8caec67b59742cac9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74387e86771e4d61a531d606f5ad3d28"
I20260619 02:55:17.436889 6100 raft_consensus.cc:3037] T 6c64880460744750b976552cb8803174 P 74387e86771e4d61a531d606f5ad3d28 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:17.436945 6233 raft_consensus.cc:3037] T 6c64880460744750b976552cb8803174 P 65ca759dd2b04dbbbdeea2f481cc29e1 [term 0 FOLLOWER]: Advancing to term 1
W20260619 02:55:17.439467 6279 tablet.cc:2369] T 6c64880460744750b976552cb8803174 P 65ca759dd2b04dbbbdeea2f481cc29e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:55:17.440299 6233 raft_consensus.cc:2454] T 6c64880460744750b976552cb8803174 P 65ca759dd2b04dbbbdeea2f481cc29e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d76e80ef0ea24f8caec67b59742cac9f in term 1.
I20260619 02:55:17.440299 6100 raft_consensus.cc:2454] T 6c64880460744750b976552cb8803174 P 74387e86771e4d61a531d606f5ad3d28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d76e80ef0ea24f8caec67b59742cac9f in term 1.
I20260619 02:55:17.440694 5902 leader_election.cc:302] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74387e86771e4d61a531d606f5ad3d28, d76e80ef0ea24f8caec67b59742cac9f; no voters:
I20260619 02:55:17.440959 6303 raft_consensus.cc:2781] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:17.441386 6303 raft_consensus.cc:686] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f [term 1 LEADER]: Becoming Leader. State: Replica: d76e80ef0ea24f8caec67b59742cac9f, State: Running, Role: LEADER
I20260619 02:55:17.441814 6303 consensus_queue.cc:237] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca759dd2b04dbbbdeea2f481cc29e1" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43273 } } peers { permanent_uuid: "74387e86771e4d61a531d606f5ad3d28" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 36717 } } peers { permanent_uuid: "d76e80ef0ea24f8caec67b59742cac9f" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 37825 } }
W20260619 02:55:17.443630 6146 tablet.cc:2369] T 6c64880460744750b976552cb8803174 P 74387e86771e4d61a531d606f5ad3d28: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:55:17.446230 5822 catalog_manager.cc:5495] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f reported cstate change: term changed from 0 to 1, leader changed from <none> to d76e80ef0ea24f8caec67b59742cac9f (127.18.134.1). New cstate: current_term: 1 leader_uuid: "d76e80ef0ea24f8caec67b59742cac9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65ca759dd2b04dbbbdeea2f481cc29e1" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43273 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74387e86771e4d61a531d606f5ad3d28" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 36717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d76e80ef0ea24f8caec67b59742cac9f" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 37825 } health_report { overall_health: HEALTHY } } }
W20260619 02:55:17.449398 6012 tablet.cc:2369] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:55:17.465940 6293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:17.460803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46068 (local address 127.18.134.1:37825)
0619 02:55:17.461023 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:17.461029 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:17.461130 (+ 101us) server_negotiation.cc:407] Connection header received
0619 02:55:17.461331 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:17.461335 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:17.461446 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:17.461568 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:55:17.461912 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.462607 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.463936 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.464337 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.464449 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:17.464899 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:55:17.465383 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:55:17.465520 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:17.465619 (+ 99us) server_negotiation.cc:299] Negotiation successful
0619 02:55:17.465700 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260619 02:55:17.474141 6233 raft_consensus.cc:1261] T 6c64880460744750b976552cb8803174 P 65ca759dd2b04dbbbdeea2f481cc29e1 [term 1 FOLLOWER]: Refusing update from remote peer d76e80ef0ea24f8caec67b59742cac9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:55:17.474150 6100 raft_consensus.cc:1261] T 6c64880460744750b976552cb8803174 P 74387e86771e4d61a531d606f5ad3d28 [term 1 FOLLOWER]: Refusing update from remote peer d76e80ef0ea24f8caec67b59742cac9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:55:17.475064 6307 consensus_queue.cc:1035] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "74387e86771e4d61a531d606f5ad3d28" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 36717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:55:17.475423 6303 consensus_queue.cc:1035] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "65ca759dd2b04dbbbdeea2f481cc29e1" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 43273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:55:17.502331 5821 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:55:17.502652 5821 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:55:17.506433 5821 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19de670d70dd4641b965d46c6bd76b76: Sending DeleteTablet for 3 replicas of tablet 6c64880460744750b976552cb8803174
I20260619 02:55:17.507556 6213 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c64880460744750b976552cb8803174 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:55:17 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:55636
I20260619 02:55:17.507606 5946 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c64880460744750b976552cb8803174 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:55:17 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:46066
I20260619 02:55:17.507606 6080 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c64880460744750b976552cb8803174 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:55:17 UTC) from {username='kudu', principal='kudu/127.18.134.62@KRBTEST.COM'} at 127.0.0.1:60506
I20260619 02:55:17.514534 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 5883
I20260619 02:55:17.520146 6321 tablet_replica.cc:330] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f: stopping tablet replica
I20260619 02:55:17.520618 6321 raft_consensus.cc:2229] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:55:17.521297 6321 raft_consensus.cc:2258] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:55:17.522079 6322 tablet_replica.cc:330] T 6c64880460744750b976552cb8803174 P 65ca759dd2b04dbbbdeea2f481cc29e1: stopping tablet replica
I20260619 02:55:17.522483 6322 raft_consensus.cc:2229] T 6c64880460744750b976552cb8803174 P 65ca759dd2b04dbbbdeea2f481cc29e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:55:17.523003 6322 raft_consensus.cc:2258] T 6c64880460744750b976552cb8803174 P 65ca759dd2b04dbbbdeea2f481cc29e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:55:17.523486 6320 tablet_replica.cc:330] T 6c64880460744750b976552cb8803174 P 74387e86771e4d61a531d606f5ad3d28: stopping tablet replica
I20260619 02:55:17.523911 6320 raft_consensus.cc:2229] T 6c64880460744750b976552cb8803174 P 74387e86771e4d61a531d606f5ad3d28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:55:17.524416 6320 raft_consensus.cc:2258] T 6c64880460744750b976552cb8803174 P 74387e86771e4d61a531d606f5ad3d28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:55:17.525306 6322 ts_tablet_manager.cc:1905] T 6c64880460744750b976552cb8803174 P 65ca759dd2b04dbbbdeea2f481cc29e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:55:17.526018 6321 ts_tablet_manager.cc:1905] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:55:17.532150 6322 ts_tablet_manager.cc:1918] T 6c64880460744750b976552cb8803174 P 65ca759dd2b04dbbbdeea2f481cc29e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:55:17.532313 6322 log.cc:1196] T 6c64880460744750b976552cb8803174 P 65ca759dd2b04dbbbdeea2f481cc29e1: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-2/wal/wals/6c64880460744750b976552cb8803174
Jun 19 02:55:17 dist-test-slave-9mr0 krb5kdc[5777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.1@KRBTEST.COM
I20260619 02:55:17.532823 6322 ts_tablet_manager.cc:1939] T 6c64880460744750b976552cb8803174 P 65ca759dd2b04dbbbdeea2f481cc29e1: Deleting consensus metadata
I20260619 02:55:17.533599 6321 ts_tablet_manager.cc:1918] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:55:17.533759 6321 log.cc:1196] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-0/wal/wals/6c64880460744750b976552cb8803174
I20260619 02:55:17.534406 6321 ts_tablet_manager.cc:1939] T 6c64880460744750b976552cb8803174 P d76e80ef0ea24f8caec67b59742cac9f: Deleting consensus metadata
I20260619 02:55:17.534837 5809 catalog_manager.cc:4841] TS 65ca759dd2b04dbbbdeea2f481cc29e1 (127.18.134.3:43273): tablet 6c64880460744750b976552cb8803174 (table test-table [id=16a5bddbcc054673bcd9d907b8542c33]) successfully deleted
I20260619 02:55:17.536535 6320 ts_tablet_manager.cc:1905] T 6c64880460744750b976552cb8803174 P 74387e86771e4d61a531d606f5ad3d28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:55:17.536707 5807 catalog_manager.cc:4841] TS d76e80ef0ea24f8caec67b59742cac9f (127.18.134.1:37825): tablet 6c64880460744750b976552cb8803174 (table test-table [id=16a5bddbcc054673bcd9d907b8542c33]) successfully deleted
I20260619 02:55:17.540230 6293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:17.517186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46072 (local address 127.18.134.1:37825)
0619 02:55:17.518063 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:17.518070 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:17.522544 (+ 4474us) server_negotiation.cc:407] Connection header received
0619 02:55:17.522838 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:17.522843 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:17.522925 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:17.523029 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:17.524823 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.525864 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.527762 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.528511 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.528630 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:17.533845 (+ 5215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:17.533859 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:17.533877 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:17.533928 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:17.537452 (+ 3524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:17.538104 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:17.538110 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:17.538113 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:17.538238 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:17.538600 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:17.538607 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:17.538609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:17.539003 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:17.539256 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:17.539587 (+ 331us) server_negotiation.cc:299] Negotiation successful
0619 02:55:17.539806 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":741}
I20260619 02:55:17.540802 6006 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:17.543068 6320 ts_tablet_manager.cc:1918] T 6c64880460744750b976552cb8803174 P 74387e86771e4d61a531d606f5ad3d28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:55:17.543200 6320 log.cc:1196] T 6c64880460744750b976552cb8803174 P 74387e86771e4d61a531d606f5ad3d28: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781837410515220-18968-0/minicluster-data/ts-1/wal/wals/6c64880460744750b976552cb8803174
I20260619 02:55:17.543608 6320 ts_tablet_manager.cc:1939] T 6c64880460744750b976552cb8803174 P 74387e86771e4d61a531d606f5ad3d28: Deleting consensus metadata
I20260619 02:55:17.545010 5809 catalog_manager.cc:4841] TS 74387e86771e4d61a531d606f5ad3d28 (127.18.134.2:36717): tablet 6c64880460744750b976552cb8803174 (table test-table [id=16a5bddbcc054673bcd9d907b8542c33]) successfully deleted
I20260619 02:55:17.645093 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 5883
I20260619 02:55:17.659698 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 6017
Jun 19 02:55:17 dist-test-slave-9mr0 krb5kdc[5777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.2@KRBTEST.COM
I20260619 02:55:17.674507 6295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:17.660423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60508 (local address 127.18.134.2:36717)
0619 02:55:17.660589 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:17.660594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:17.660613 (+ 19us) server_negotiation.cc:407] Connection header received
0619 02:55:17.660676 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:17.660680 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:17.660742 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:17.660830 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:17.662053 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.662843 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.664098 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.664489 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.664605 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:17.669140 (+ 4535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:17.669156 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:17.669172 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:17.669215 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:17.671960 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:17.672639 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:17.672646 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:17.672649 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:17.672789 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:17.673155 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:17.673160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:17.673163 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:17.673528 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:17.673760 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:17.674050 (+ 290us) server_negotiation.cc:299] Negotiation successful
0619 02:55:17.674255 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 02:55:17.675168 6140 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:17.762277 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 6017
I20260619 02:55:17.776461 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 6150
Jun 19 02:55:17 dist-test-slave-9mr0 krb5kdc[5777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.3@KRBTEST.COM
I20260619 02:55:17.791903 6291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:17.777330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55648 (local address 127.18.134.3:43273)
0619 02:55:17.777511 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:17.777516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:17.777536 (+ 20us) server_negotiation.cc:407] Connection header received
0619 02:55:17.777679 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:17.777683 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:17.777752 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:17.777861 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:17.779479 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.780213 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.781593 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:17.782004 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:17.782111 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:17.786473 (+ 4362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:17.786490 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:17.786507 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:17.786549 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:17.789248 (+ 2699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:17.789957 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:17.789964 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:17.789966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:17.790149 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:17.790481 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:17.790484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:17.790486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:17.790939 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:17.791202 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:17.791491 (+ 289us) server_negotiation.cc:299] Negotiation successful
0619 02:55:17.791665 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 02:55:17.792546 6273 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:17.880395 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 6150
I20260619 02:55:17.893594 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 5790
I20260619 02:55:17.894774 5852 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:18.002909 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 5790
2026-06-19T02:55:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4069 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-19T02:55:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:55:18Z Disabled control of system clock
I20260619 02:55:18.070544 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:44715
--webserver_interface=127.18.134.62
--webserver_port=0
--builtin_ntp_servers=127.18.134.20:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:44715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:55:18.384490 6331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:18.384752 6331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:18.384871 6331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:18.392078 6331 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:18.392187 6331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:18.392246 6331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:18.392325 6331 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:18.392383 6331 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:18.401743 6331 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:44715
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:44715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:18.403033 6331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:18.404781 6331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:18.416033 6336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:18.415944 6339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:18.415975 6337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:18.416657 6331 server_base.cc:1034] running on GCE node
I20260619 02:55:18.417444 6331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:18.418855 6331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:18.420059 6331 hybrid_clock.cc:648] HybridClock initialized: now 1781837718420005 us; error 47 us; skew 500 ppm
I20260619 02:55:18.420419 6331 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:55:18.422600 6331 webserver.cc:460] Webserver started at http://127.18.134.62:35173/ using document root <none> and password file <none>
I20260619 02:55:18.423300 6331 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:18.423385 6331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:18.423689 6331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:18.426496 6331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "f95f051ea5a14afc8c3074ed214f0612"
format_stamp: "Formatted at 2026-06-19 02:55:18 on dist-test-slave-9mr0"
server_key: "e0e3c0b838d107b637c0c33f9a9a36c2"
server_key_iv: "5fcdeff955108e12f7134ded032f7e1e"
server_key_version: "encryptionkey@0"
I20260619 02:55:18.427292 6331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "f95f051ea5a14afc8c3074ed214f0612"
format_stamp: "Formatted at 2026-06-19 02:55:18 on dist-test-slave-9mr0"
server_key: "e0e3c0b838d107b637c0c33f9a9a36c2"
server_key_iv: "5fcdeff955108e12f7134ded032f7e1e"
server_key_version: "encryptionkey@0"
I20260619 02:55:18.432318 6331 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 02:55:18.435702 6345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:18.437083 6331 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 02:55:18.437244 6331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "f95f051ea5a14afc8c3074ed214f0612"
format_stamp: "Formatted at 2026-06-19 02:55:18 on dist-test-slave-9mr0"
server_key: "e0e3c0b838d107b637c0c33f9a9a36c2"
server_key_iv: "5fcdeff955108e12f7134ded032f7e1e"
server_key_version: "encryptionkey@0"
I20260619 02:55:18.437435 6331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:18.463563 6331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:18.464449 6331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:18.464759 6331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:18.482131 6331 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:44715
I20260619 02:55:18.482167 6396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:44715 every 8 connection(s)
I20260619 02:55:18.483393 6331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:55:18.487140 6397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:18.491860 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 6331
I20260619 02:55:18.492041 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:55:18.492578 18968 external_mini_cluster.cc:1428] Setting key cac9ea9212fb2d9c1deae915b0b01ce8
I20260619 02:55:18.498129 6397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f95f051ea5a14afc8c3074ed214f0612: Bootstrap starting.
I20260619 02:55:18.501174 6397 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f95f051ea5a14afc8c3074ed214f0612: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:18.502203 6397 log.cc:824] T 00000000000000000000000000000000 P f95f051ea5a14afc8c3074ed214f0612: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:18.505404 6397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f95f051ea5a14afc8c3074ed214f0612: No bootstrap required, opened a new log
I20260619 02:55:18.506114 6400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:18.494501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45316 (local address 127.18.134.62:44715)
0619 02:55:18.495530 (+ 1029us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:18.495544 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:18.495588 (+ 44us) server_negotiation.cc:407] Connection header received
0619 02:55:18.497299 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:18.497321 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:18.497771 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:18.498130 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:18.498700 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:18.500001 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:18.501311 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:18.501989 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:18.502168 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:18.503358 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:18.503365 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:18.503378 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:18.504128 (+ 750us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:18.504225 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:18.504251 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:18.504373 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:18.504572 (+ 199us) server_negotiation.cc:299] Negotiation successful
0619 02:55:18.504904 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":341,"threads_started":1}
I20260619 02:55:18.512800 6397 raft_consensus.cc:348] T 00000000000000000000000000000000 P f95f051ea5a14afc8c3074ed214f0612 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95f051ea5a14afc8c3074ed214f0612" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 44715 } }
I20260619 02:55:18.513123 6397 raft_consensus.cc:374] T 00000000000000000000000000000000 P f95f051ea5a14afc8c3074ed214f0612 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:18.513193 6397 raft_consensus.cc:729] T 00000000000000000000000000000000 P f95f051ea5a14afc8c3074ed214f0612 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f95f051ea5a14afc8c3074ed214f0612, State: Initialized, Role: FOLLOWER
I20260619 02:55:18.513659 6397 consensus_queue.cc:260] T 00000000000000000000000000000000 P f95f051ea5a14afc8c3074ed214f0612 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95f051ea5a14afc8c3074ed214f0612" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 44715 } }
I20260619 02:55:18.513860 6397 raft_consensus.cc:388] T 00000000000000000000000000000000 P f95f051ea5a14afc8c3074ed214f0612 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:55:18.513950 6397 raft_consensus.cc:482] T 00000000000000000000000000000000 P f95f051ea5a14afc8c3074ed214f0612 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:55:18.514070 6397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f95f051ea5a14afc8c3074ed214f0612 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:18.516212 6397 raft_consensus.cc:504] T 00000000000000000000000000000000 P f95f051ea5a14afc8c3074ed214f0612 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95f051ea5a14afc8c3074ed214f0612" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 44715 } }
I20260619 02:55:18.516665 6397 leader_election.cc:302] T 00000000000000000000000000000000 P f95f051ea5a14afc8c3074ed214f0612 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f95f051ea5a14afc8c3074ed214f0612; no voters:
I20260619 02:55:18.517174 6397 leader_election.cc:288] T 00000000000000000000000000000000 P f95f051ea5a14afc8c3074ed214f0612 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:55:18.517319 6402 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f95f051ea5a14afc8c3074ed214f0612 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:18.517941 6402 raft_consensus.cc:686] T 00000000000000000000000000000000 P f95f051ea5a14afc8c3074ed214f0612 [term 1 LEADER]: Becoming Leader. State: Replica: f95f051ea5a14afc8c3074ed214f0612, State: Running, Role: LEADER
I20260619 02:55:18.518839 6402 consensus_queue.cc:237] T 00000000000000000000000000000000 P f95f051ea5a14afc8c3074ed214f0612 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95f051ea5a14afc8c3074ed214f0612" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 44715 } }
I20260619 02:55:18.519217 6397 sys_catalog.cc:564] T 00000000000000000000000000000000 P f95f051ea5a14afc8c3074ed214f0612 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:55:18.521607 6403 sys_catalog.cc:455] T 00000000000000000000000000000000 P f95f051ea5a14afc8c3074ed214f0612 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f95f051ea5a14afc8c3074ed214f0612" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95f051ea5a14afc8c3074ed214f0612" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 44715 } } }
I20260619 02:55:18.522015 6403 sys_catalog.cc:458] T 00000000000000000000000000000000 P f95f051ea5a14afc8c3074ed214f0612 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:18.522444 6404 sys_catalog.cc:455] T 00000000000000000000000000000000 P f95f051ea5a14afc8c3074ed214f0612 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f95f051ea5a14afc8c3074ed214f0612. Latest consensus state: current_term: 1 leader_uuid: "f95f051ea5a14afc8c3074ed214f0612" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95f051ea5a14afc8c3074ed214f0612" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 44715 } } }
I20260619 02:55:18.522778 6404 sys_catalog.cc:458] T 00000000000000000000000000000000 P f95f051ea5a14afc8c3074ed214f0612 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:18.526286 6411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:55:18.530478 6411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:55:18.538416 6411 catalog_manager.cc:1347] Generated new cluster ID: 046c1f8b5204463894f9db4f3c258e83
I20260619 02:55:18.538533 6411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:55:18.552888 6411 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:55:18.554085 6411 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:55:18.564064 6411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f95f051ea5a14afc8c3074ed214f0612: Generated new TSK 0
I20260619 02:55:18.565119 6411 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:55:18.577603 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.1:0
--local_ip_for_outbound_sockets=127.18.134.1
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:44715
--builtin_ntp_servers=127.18.134.20:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 02:55:18.895393 6421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:18.895692 6421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:18.895823 6421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:18.903076 6421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:18.903184 6421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:18.903324 6421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.1
I20260619 02:55:18.912328 6421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.134.1
--webserver_port=0
--tserver_master_addrs=127.18.134.62:44715
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.1
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:18.913615 6421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:18.915318 6421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:18.927914 6426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:18.927955 6429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:18.927878 6427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:18.928750 6421 server_base.cc:1034] running on GCE node
I20260619 02:55:18.929368 6421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:18.930287 6421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:18.931499 6421 hybrid_clock.cc:648] HybridClock initialized: now 1781837718931445 us; error 33 us; skew 500 ppm
I20260619 02:55:18.931809 6421 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:55:18.933986 6421 webserver.cc:460] Webserver started at http://127.18.134.1:43419/ using document root <none> and password file <none>
I20260619 02:55:18.934721 6421 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:18.934829 6421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:18.935139 6421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:18.937992 6421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-0/data/instance:
uuid: "96569cbe15ee461ea960a8ba648d2b19"
format_stamp: "Formatted at 2026-06-19 02:55:18 on dist-test-slave-9mr0"
server_key: "0e199937deb633f04c6dd0141d76e9f9"
server_key_iv: "226835ea43deec93aa95db27fa6cc8f8"
server_key_version: "encryptionkey@0"
I20260619 02:55:18.938796 6421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance:
uuid: "96569cbe15ee461ea960a8ba648d2b19"
format_stamp: "Formatted at 2026-06-19 02:55:18 on dist-test-slave-9mr0"
server_key: "0e199937deb633f04c6dd0141d76e9f9"
server_key_iv: "226835ea43deec93aa95db27fa6cc8f8"
server_key_version: "encryptionkey@0"
I20260619 02:55:18.943858 6421 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 02:55:18.947201 6435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:18.948530 6421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:55:18.948686 6421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-0/wal
uuid: "96569cbe15ee461ea960a8ba648d2b19"
format_stamp: "Formatted at 2026-06-19 02:55:18 on dist-test-slave-9mr0"
server_key: "0e199937deb633f04c6dd0141d76e9f9"
server_key_iv: "226835ea43deec93aa95db27fa6cc8f8"
server_key_version: "encryptionkey@0"
I20260619 02:55:18.948896 6421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:18.969741 6421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:18.970548 6421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:18.970860 6421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:18.971789 6421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:55:18.973193 6421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:55:18.973285 6421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:18.973384 6421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:55:18.973455 6421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:19.002687 6421 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.1:33867
I20260619 02:55:19.002768 6547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.1:33867 every 8 connection(s)
I20260619 02:55:19.003937 6421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-0/data/info.pb
I20260619 02:55:19.012866 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 6421
I20260619 02:55:19.013016 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-0/wal/instance
I20260619 02:55:19.013417 18968 external_mini_cluster.cc:1428] Setting key 2433b31df49c19da6647fa3e375cc3d3
I20260619 02:55:19.015861 6400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:19.006416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.1:35169 (local address 127.18.134.62:44715)
0619 02:55:19.006582 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:19.006587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:19.007918 (+ 1331us) server_negotiation.cc:407] Connection header received
0619 02:55:19.009631 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:19.009636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:19.009713 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:19.009800 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:19.010861 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:19.011561 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:19.013117 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:19.013532 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:19.013664 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:19.015134 (+ 1470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:19.015140 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:19.015144 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:19.015166 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:19.015209 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:19.015224 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:19.015321 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:19.015550 (+ 229us) server_negotiation.cc:299] Negotiation successful
0619 02:55:19.015629 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 02:55:19.016377 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.2:0
--local_ip_for_outbound_sockets=127.18.134.2
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:44715
--builtin_ntp_servers=127.18.134.20:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 02:55:19.017262 6550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:19.006939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:44715 (local address 127.18.134.1:35169)
0619 02:55:19.007771 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:19.007808 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:19.009368 (+ 1560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:19.010028 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:19.010036 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:19.010127 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:19.010703 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:19.010723 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:19.011722 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:19.011726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:19.012966 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:19.012981 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:19.013753 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:19.013760 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:19.013998 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:19.014793 (+ 795us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:55:19.014796 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:19.014836 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:55:19.014922 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:19.015326 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:19.015338 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:19.015346 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 02:55:19.015530 (+ 184us) client_negotiation.cc:240] Negotiation successful
0619 02:55:19.015934 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":251,"threads_started":1}
I20260619 02:55:19.018704 6548 heartbeater.cc:347] Connected to a master server at 127.18.134.62:44715
I20260619 02:55:19.019055 6548 heartbeater.cc:464] Registering TS with master...
I20260619 02:55:19.019905 6548 heartbeater.cc:511] Master 127.18.134.62:44715 requested a full tablet report, sending...
I20260619 02:55:19.021908 6362 ts_manager.cc:194] Registered new tserver with Master: 96569cbe15ee461ea960a8ba648d2b19 (127.18.134.1:33867)
I20260619 02:55:19.023520 6362 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.134.1:35169
W20260619 02:55:19.328523 6551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:19.328784 6551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:19.328905 6551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:19.336038 6551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:19.336146 6551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:19.336274 6551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.2
I20260619 02:55:19.345374 6551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.134.2
--webserver_port=0
--tserver_master_addrs=127.18.134.62:44715
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.2
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:19.346612 6551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:19.348394 6551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:19.361287 6556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:19.361222 6557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:19.361254 6559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:19.362134 6551 server_base.cc:1034] running on GCE node
I20260619 02:55:19.362793 6551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:19.363709 6551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:19.364938 6551 hybrid_clock.cc:648] HybridClock initialized: now 1781837719364900 us; error 26 us; skew 500 ppm
I20260619 02:55:19.365257 6551 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:55:19.367489 6551 webserver.cc:460] Webserver started at http://127.18.134.2:34837/ using document root <none> and password file <none>
I20260619 02:55:19.368160 6551 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:19.368252 6551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:19.368546 6551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:19.371414 6551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-1/data/instance:
uuid: "72f71fef589a46be900e77e2928c1498"
format_stamp: "Formatted at 2026-06-19 02:55:19 on dist-test-slave-9mr0"
server_key: "e23f7b5d9b9d9dd0ce23855c97c96252"
server_key_iv: "059e7bf2d550b75d74c9909fdaa78544"
server_key_version: "encryptionkey@0"
I20260619 02:55:19.372159 6551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance:
uuid: "72f71fef589a46be900e77e2928c1498"
format_stamp: "Formatted at 2026-06-19 02:55:19 on dist-test-slave-9mr0"
server_key: "e23f7b5d9b9d9dd0ce23855c97c96252"
server_key_iv: "059e7bf2d550b75d74c9909fdaa78544"
server_key_version: "encryptionkey@0"
I20260619 02:55:19.377118 6551 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:55:19.380533 6565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:19.381850 6551 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 02:55:19.381987 6551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-1/wal
uuid: "72f71fef589a46be900e77e2928c1498"
format_stamp: "Formatted at 2026-06-19 02:55:19 on dist-test-slave-9mr0"
server_key: "e23f7b5d9b9d9dd0ce23855c97c96252"
server_key_iv: "059e7bf2d550b75d74c9909fdaa78544"
server_key_version: "encryptionkey@0"
I20260619 02:55:19.382225 6551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:19.407477 6551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:19.408308 6551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:19.408592 6551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:19.409583 6551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:55:19.411072 6551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:55:19.411157 6551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:19.411252 6551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:55:19.411319 6551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:19.440318 6551 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.2:34107
I20260619 02:55:19.440363 6677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.2:34107 every 8 connection(s)
I20260619 02:55:19.441571 6551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-1/data/info.pb
I20260619 02:55:19.450156 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 6551
I20260619 02:55:19.450314 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-1/wal/instance
I20260619 02:55:19.450778 18968 external_mini_cluster.cc:1428] Setting key c8155177b1b7b7fae409af76bde34878
I20260619 02:55:19.453235 6400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:19.443913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:51655 (local address 127.18.134.62:44715)
0619 02:55:19.444132 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:19.444138 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:19.445405 (+ 1267us) server_negotiation.cc:407] Connection header received
0619 02:55:19.447140 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:19.447144 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:19.447209 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:19.447286 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:19.448218 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:19.448887 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:19.450342 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:19.450811 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:19.450927 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:19.452431 (+ 1504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:19.452438 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:19.452442 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:19.452463 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:19.452525 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:19.452542 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:19.452630 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:19.452893 (+ 263us) server_negotiation.cc:299] Negotiation successful
0619 02:55:19.453002 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260619 02:55:19.453624 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.134.3:0
--local_ip_for_outbound_sockets=127.18.134.3
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:44715
--builtin_ntp_servers=127.18.134.20:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 02:55:19.454519 6680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:19.444431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:44715 (local address 127.18.134.2:51655)
0619 02:55:19.445259 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:19.445294 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:19.446892 (+ 1598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:19.447454 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:19.447461 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:19.447550 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:19.448088 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:19.448104 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:19.449000 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:19.449003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:19.450200 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:19.450211 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:19.451144 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:19.451149 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:19.451406 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:19.452071 (+ 665us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:55:19.452073 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:19.452128 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:55:19.452243 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:19.452654 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:19.452664 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:19.452671 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 02:55:19.452849 (+ 178us) client_negotiation.cc:240] Negotiation successful
0619 02:55:19.453244 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":242,"threads_started":1}
I20260619 02:55:19.455857 6678 heartbeater.cc:347] Connected to a master server at 127.18.134.62:44715
I20260619 02:55:19.456144 6678 heartbeater.cc:464] Registering TS with master...
I20260619 02:55:19.456938 6678 heartbeater.cc:511] Master 127.18.134.62:44715 requested a full tablet report, sending...
I20260619 02:55:19.458305 6362 ts_manager.cc:194] Registered new tserver with Master: 72f71fef589a46be900e77e2928c1498 (127.18.134.2:34107)
I20260619 02:55:19.459149 6362 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.134.2:51655
W20260619 02:55:19.760800 6681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:19.761051 6681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:19.761170 6681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:19.768239 6681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:19.768357 6681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:19.768517 6681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.134.3
I20260619 02:55:19.777359 6681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.134.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.134.3
--webserver_port=0
--tserver_master_addrs=127.18.134.62:44715
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.134.3
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:19.778573 6681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:19.780318 6681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:19.792681 6686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:19.792658 6687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:19.792670 6689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:19.793279 6681 server_base.cc:1034] running on GCE node
I20260619 02:55:19.793865 6681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:19.794816 6681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:19.796008 6681 hybrid_clock.cc:648] HybridClock initialized: now 1781837719795951 us; error 54 us; skew 500 ppm
I20260619 02:55:19.796301 6681 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 02:55:19.798496 6681 webserver.cc:460] Webserver started at http://127.18.134.3:37067/ using document root <none> and password file <none>
I20260619 02:55:19.799178 6681 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:19.799279 6681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:19.799602 6681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:19.802325 6681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-2/data/instance:
uuid: "0c2f0374e7424fe3ba993ae52a7d9d8d"
format_stamp: "Formatted at 2026-06-19 02:55:19 on dist-test-slave-9mr0"
server_key: "8fd20dae302440bd26e66cfa53750f11"
server_key_iv: "506201631ea278708747e7e26fd428b6"
server_key_version: "encryptionkey@0"
I20260619 02:55:19.803089 6681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance:
uuid: "0c2f0374e7424fe3ba993ae52a7d9d8d"
format_stamp: "Formatted at 2026-06-19 02:55:19 on dist-test-slave-9mr0"
server_key: "8fd20dae302440bd26e66cfa53750f11"
server_key_iv: "506201631ea278708747e7e26fd428b6"
server_key_version: "encryptionkey@0"
I20260619 02:55:19.807907 6681 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260619 02:55:19.811133 6695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:19.812361 6681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 02:55:19.812508 6681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-2/wal
uuid: "0c2f0374e7424fe3ba993ae52a7d9d8d"
format_stamp: "Formatted at 2026-06-19 02:55:19 on dist-test-slave-9mr0"
server_key: "8fd20dae302440bd26e66cfa53750f11"
server_key_iv: "506201631ea278708747e7e26fd428b6"
server_key_version: "encryptionkey@0"
I20260619 02:55:19.812711 6681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:19.845964 6681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:19.846840 6681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:19.847122 6681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:19.848100 6681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 02:55:19.849551 6681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 02:55:19.849659 6681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:19.849772 6681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 02:55:19.849841 6681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:19.879551 6681 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.3:45569
I20260619 02:55:19.879630 6807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.3:45569 every 8 connection(s)
I20260619 02:55:19.880888 6681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-2/data/info.pb
I20260619 02:55:19.886356 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 6681
I20260619 02:55:19.886509 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-2/wal/instance
I20260619 02:55:19.887042 18968 external_mini_cluster.cc:1428] Setting key a5f827841a0e6a970ccc46d0795f253b
I20260619 02:55:19.893255 6400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:19.883384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.3:51953 (local address 127.18.134.62:44715)
0619 02:55:19.883566 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:19.883570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:19.884881 (+ 1311us) server_negotiation.cc:407] Connection header received
0619 02:55:19.886836 (+ 1955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:19.886842 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:19.886928 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:19.887039 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:19.888268 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:19.889192 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:19.890942 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:19.891328 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:19.891421 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:19.892616 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:19.892620 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:19.892624 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:19.892641 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:19.892668 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:19.892678 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:19.892739 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:19.892958 (+ 219us) server_negotiation.cc:299] Negotiation successful
0619 02:55:19.893028 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 02:55:19.894232 6810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:19.883830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:44715 (local address 127.18.134.3:51953)
0619 02:55:19.884724 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:19.884760 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:19.886508 (+ 1748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:19.887273 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:19.887281 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:19.887367 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:19.888087 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:19.888113 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:19.889352 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:19.889358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:19.890817 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:19.890834 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:19.891527 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:19.891532 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:19.891719 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:19.892345 (+ 626us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 02:55:19.892347 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:19.892384 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 02:55:19.892470 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:19.892783 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:19.892792 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:19.892799 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 02:55:19.892953 (+ 154us) client_negotiation.cc:240] Negotiation successful
0619 02:55:19.893239 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":287,"threads_started":1}
I20260619 02:55:19.895463 6808 heartbeater.cc:347] Connected to a master server at 127.18.134.62:44715
I20260619 02:55:19.895740 6808 heartbeater.cc:464] Registering TS with master...
I20260619 02:55:19.896420 6808 heartbeater.cc:511] Master 127.18.134.62:44715 requested a full tablet report, sending...
I20260619 02:55:19.897692 6362 ts_manager.cc:194] Registered new tserver with Master: 0c2f0374e7424fe3ba993ae52a7d9d8d (127.18.134.3:45569)
I20260619 02:55:19.898465 6362 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.134.3:51953
I20260619 02:55:19.901909 18968 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 02:55:19.911512 6400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:19.905781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55832 (local address 127.18.134.62:44715)
0619 02:55:19.905959 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:19.905963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:19.906399 (+ 436us) server_negotiation.cc:407] Connection header received
0619 02:55:19.906551 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:19.906556 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:19.906690 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:19.906789 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:19.907231 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:19.907991 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:19.909317 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:19.909732 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:19.909835 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:19.910940 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:19.910947 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:19.910952 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:19.910972 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:19.911006 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:19.911019 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:19.911089 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:19.911178 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 02:55:19.911255 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 02:55:19.915172 6362 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 02:55:19.921970 6362 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 02:55:19.945115 6821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:19.933691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.2:34107 (local address 127.0.0.1:52374)
0619 02:55:19.936133 (+ 2442us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:19.936183 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:19.936369 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:19.937609 (+ 1240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:19.937620 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:19.937658 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:19.938048 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:19.938060 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:19.939758 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:19.939763 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:19.942205 (+ 2442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:19.942220 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:19.944077 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:19.944084 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:19.944341 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:19.944400 (+ 59us) client_negotiation.cc:769] Sending connection context
0619 02:55:19.944537 (+ 137us) client_negotiation.cc:240] Negotiation successful
0619 02:55:19.944723 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2142,"mutex_wait_us":22,"thread_start_us":493,"threads_started":1}
I20260619 02:55:19.946020 6820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:19.936128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:33867 (local address 127.0.0.1:55146)
0619 02:55:19.938773 (+ 2645us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:19.938793 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:19.939040 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:19.939659 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:19.939663 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:19.939688 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:19.940075 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:19.940104 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:19.941760 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:19.941765 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:19.943517 (+ 1752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:19.943529 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:19.945107 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:19.945114 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:19.945467 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:19.945501 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 02:55:19.945587 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 02:55:19.945716 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2375,"thread_start_us":1251,"threads_started":1}
I20260619 02:55:19.946127 6822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:19.933946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52374 (local address 127.18.134.2:34107)
0619 02:55:19.936964 (+ 3018us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:19.936974 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:19.937005 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:55:19.937108 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:19.937116 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:19.937383 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:19.937578 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:19.938325 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:19.939602 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:19.942425 (+ 2823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:19.943884 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:19.944016 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:19.944279 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:19.945381 (+ 1102us) server_negotiation.cc:299] Negotiation successful
0619 02:55:19.945743 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2837,"thread_start_us":2702,"threads_started":1}
I20260619 02:55:19.946892 6825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:19.936643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55146 (local address 127.18.134.1:33867)
0619 02:55:19.937229 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:19.937238 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:19.938935 (+ 1697us) server_negotiation.cc:407] Connection header received
0619 02:55:19.939237 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:19.939244 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:19.939464 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:19.939655 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:19.940274 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:19.941620 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:19.943685 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:19.944900 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:19.945040 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:19.945259 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:19.946376 (+ 1117us) server_negotiation.cc:299] Negotiation successful
0619 02:55:19.946547 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":323,"threads_started":1}
I20260619 02:55:19.948149 6823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:19.934412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58976 (local address 127.18.134.3:45569)
0619 02:55:19.939201 (+ 4789us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:19.939216 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:19.939246 (+ 30us) server_negotiation.cc:407] Connection header received
0619 02:55:19.939326 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:19.939332 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:19.939491 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:19.939656 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:19.940609 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:19.942127 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:19.944555 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:19.945687 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:19.945800 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:19.946036 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:19.947039 (+ 1003us) server_negotiation.cc:299] Negotiation successful
0619 02:55:19.947912 (+ 873us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4629,"thread_start_us":4489,"threads_started":1}
I20260619 02:55:19.946919 6824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:19.933881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:45569 (local address 127.0.0.1:58976)
0619 02:55:19.937815 (+ 3934us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:19.937833 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:19.937946 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:19.939904 (+ 1958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:19.939912 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:19.939959 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:19.940402 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:19.940414 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:19.942304 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:19.942308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:19.944390 (+ 2082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:19.944402 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:19.946164 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:19.946169 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:19.946328 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:19.946357 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 02:55:19.946435 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 02:55:19.946550 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3632,"mutex_wait_us":48,"thread_start_us":1816,"threads_started":1}
I20260619 02:55:19.949982 6613 tablet_service.cc:1467] Processing CreateTablet for tablet 619716c0cd0146759c863ad3e762800b (DEFAULT_TABLE table=test-table [id=b75ffd4cd826410ca2f253f96834bd76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:55:19.950404 6483 tablet_service.cc:1467] Processing CreateTablet for tablet 619716c0cd0146759c863ad3e762800b (DEFAULT_TABLE table=test-table [id=b75ffd4cd826410ca2f253f96834bd76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:55:19.950991 6743 tablet_service.cc:1467] Processing CreateTablet for tablet 619716c0cd0146759c863ad3e762800b (DEFAULT_TABLE table=test-table [id=b75ffd4cd826410ca2f253f96834bd76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 02:55:19.951661 6613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 619716c0cd0146759c863ad3e762800b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:19.951658 6483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 619716c0cd0146759c863ad3e762800b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:19.952327 6743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 619716c0cd0146759c863ad3e762800b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:19.963346 6826 tablet_bootstrap.cc:492] T 619716c0cd0146759c863ad3e762800b P 0c2f0374e7424fe3ba993ae52a7d9d8d: Bootstrap starting.
I20260619 02:55:19.967330 6828 tablet_bootstrap.cc:492] T 619716c0cd0146759c863ad3e762800b P 96569cbe15ee461ea960a8ba648d2b19: Bootstrap starting.
I20260619 02:55:19.967437 6826 tablet_bootstrap.cc:654] T 619716c0cd0146759c863ad3e762800b P 0c2f0374e7424fe3ba993ae52a7d9d8d: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:19.968146 6827 tablet_bootstrap.cc:492] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498: Bootstrap starting.
I20260619 02:55:19.968719 6826 log.cc:824] T 619716c0cd0146759c863ad3e762800b P 0c2f0374e7424fe3ba993ae52a7d9d8d: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:19.971372 6828 tablet_bootstrap.cc:654] T 619716c0cd0146759c863ad3e762800b P 96569cbe15ee461ea960a8ba648d2b19: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:19.971400 6827 tablet_bootstrap.cc:654] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:19.972311 6827 log.cc:824] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:19.972512 6828 log.cc:824] T 619716c0cd0146759c863ad3e762800b P 96569cbe15ee461ea960a8ba648d2b19: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:19.973420 6826 tablet_bootstrap.cc:492] T 619716c0cd0146759c863ad3e762800b P 0c2f0374e7424fe3ba993ae52a7d9d8d: No bootstrap required, opened a new log
I20260619 02:55:19.973762 6826 ts_tablet_manager.cc:1397] T 619716c0cd0146759c863ad3e762800b P 0c2f0374e7424fe3ba993ae52a7d9d8d: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260619 02:55:19.977814 6827 tablet_bootstrap.cc:492] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498: No bootstrap required, opened a new log
I20260619 02:55:19.977814 6828 tablet_bootstrap.cc:492] T 619716c0cd0146759c863ad3e762800b P 96569cbe15ee461ea960a8ba648d2b19: No bootstrap required, opened a new log
I20260619 02:55:19.978106 6827 ts_tablet_manager.cc:1397] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260619 02:55:19.978106 6828 ts_tablet_manager.cc:1397] T 619716c0cd0146759c863ad3e762800b P 96569cbe15ee461ea960a8ba648d2b19: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260619 02:55:19.981454 6826 raft_consensus.cc:348] T 619716c0cd0146759c863ad3e762800b P 0c2f0374e7424fe3ba993ae52a7d9d8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f71fef589a46be900e77e2928c1498" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 34107 } } peers { permanent_uuid: "0c2f0374e7424fe3ba993ae52a7d9d8d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 45569 } } peers { permanent_uuid: "96569cbe15ee461ea960a8ba648d2b19" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33867 } }
I20260619 02:55:19.981817 6826 raft_consensus.cc:374] T 619716c0cd0146759c863ad3e762800b P 0c2f0374e7424fe3ba993ae52a7d9d8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:19.981926 6826 raft_consensus.cc:729] T 619716c0cd0146759c863ad3e762800b P 0c2f0374e7424fe3ba993ae52a7d9d8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c2f0374e7424fe3ba993ae52a7d9d8d, State: Initialized, Role: FOLLOWER
I20260619 02:55:19.982501 6826 consensus_queue.cc:260] T 619716c0cd0146759c863ad3e762800b P 0c2f0374e7424fe3ba993ae52a7d9d8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f71fef589a46be900e77e2928c1498" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 34107 } } peers { permanent_uuid: "0c2f0374e7424fe3ba993ae52a7d9d8d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 45569 } } peers { permanent_uuid: "96569cbe15ee461ea960a8ba648d2b19" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33867 } }
I20260619 02:55:19.983564 6808 heartbeater.cc:503] Master 127.18.134.62:44715 was elected leader, sending a full tablet report...
I20260619 02:55:19.984258 6826 ts_tablet_manager.cc:1428] T 619716c0cd0146759c863ad3e762800b P 0c2f0374e7424fe3ba993ae52a7d9d8d: Time spent starting tablet: real 0.010s user 0.006s sys 0.003s
I20260619 02:55:19.987987 6828 raft_consensus.cc:348] T 619716c0cd0146759c863ad3e762800b P 96569cbe15ee461ea960a8ba648d2b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f71fef589a46be900e77e2928c1498" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 34107 } } peers { permanent_uuid: "0c2f0374e7424fe3ba993ae52a7d9d8d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 45569 } } peers { permanent_uuid: "96569cbe15ee461ea960a8ba648d2b19" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33867 } }
I20260619 02:55:19.988348 6828 raft_consensus.cc:374] T 619716c0cd0146759c863ad3e762800b P 96569cbe15ee461ea960a8ba648d2b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:19.988437 6828 raft_consensus.cc:729] T 619716c0cd0146759c863ad3e762800b P 96569cbe15ee461ea960a8ba648d2b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96569cbe15ee461ea960a8ba648d2b19, State: Initialized, Role: FOLLOWER
I20260619 02:55:19.989049 6828 consensus_queue.cc:260] T 619716c0cd0146759c863ad3e762800b P 96569cbe15ee461ea960a8ba648d2b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f71fef589a46be900e77e2928c1498" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 34107 } } peers { permanent_uuid: "0c2f0374e7424fe3ba993ae52a7d9d8d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 45569 } } peers { permanent_uuid: "96569cbe15ee461ea960a8ba648d2b19" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33867 } }
I20260619 02:55:19.989733 6827 raft_consensus.cc:348] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f71fef589a46be900e77e2928c1498" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 34107 } } peers { permanent_uuid: "0c2f0374e7424fe3ba993ae52a7d9d8d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 45569 } } peers { permanent_uuid: "96569cbe15ee461ea960a8ba648d2b19" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33867 } }
I20260619 02:55:19.990078 6548 heartbeater.cc:503] Master 127.18.134.62:44715 was elected leader, sending a full tablet report...
I20260619 02:55:19.990202 6827 raft_consensus.cc:374] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:19.990295 6827 raft_consensus.cc:729] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72f71fef589a46be900e77e2928c1498, State: Initialized, Role: FOLLOWER
I20260619 02:55:19.990787 6828 ts_tablet_manager.cc:1428] T 619716c0cd0146759c863ad3e762800b P 96569cbe15ee461ea960a8ba648d2b19: Time spent starting tablet: real 0.013s user 0.012s sys 0.003s
I20260619 02:55:19.990813 6827 consensus_queue.cc:260] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f71fef589a46be900e77e2928c1498" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 34107 } } peers { permanent_uuid: "0c2f0374e7424fe3ba993ae52a7d9d8d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 45569 } } peers { permanent_uuid: "96569cbe15ee461ea960a8ba648d2b19" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33867 } }
I20260619 02:55:19.991766 6678 heartbeater.cc:503] Master 127.18.134.62:44715 was elected leader, sending a full tablet report...
I20260619 02:55:19.992383 6827 ts_tablet_manager.cc:1428] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260619 02:55:19.999018 6834 raft_consensus.cc:482] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:55:19.999258 6834 raft_consensus.cc:504] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f71fef589a46be900e77e2928c1498" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 34107 } } peers { permanent_uuid: "0c2f0374e7424fe3ba993ae52a7d9d8d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 45569 } } peers { permanent_uuid: "96569cbe15ee461ea960a8ba648d2b19" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33867 } }
I20260619 02:55:20.000713 6834 leader_election.cc:288] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c2f0374e7424fe3ba993ae52a7d9d8d (127.18.134.3:45569), 96569cbe15ee461ea960a8ba648d2b19 (127.18.134.1:33867)
W20260619 02:55:20.006140 6549 tablet.cc:2369] T 619716c0cd0146759c863ad3e762800b P 96569cbe15ee461ea960a8ba648d2b19: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:55:20.007807 6823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:20.001028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:34155 (local address 127.18.134.3:45569)
0619 02:55:20.001247 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:20.001253 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:20.001626 (+ 373us) server_negotiation.cc:407] Connection header received
0619 02:55:20.001852 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:20.001857 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:20.001928 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:20.001985 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:20.002757 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:20.003872 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:20.005848 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:20.006963 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:20.007124 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:20.007196 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:20.007435 (+ 239us) server_negotiation.cc:299] Negotiation successful
0619 02:55:20.007572 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 02:55:20.007874 6835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:20.000964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.3:45569 (local address 127.18.134.2:34155)
0619 02:55:20.001510 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:20.001525 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:20.001680 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:20.002130 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:20.002134 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:20.002154 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:20.002517 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:20.002529 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:20.004026 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:20.004031 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:20.005672 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:20.005683 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:20.007132 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:20.007138 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:20.007302 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:20.007324 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 02:55:20.007400 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 02:55:20.007521 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":50,"thread_start_us":266,"threads_started":1}
I20260619 02:55:20.008739 6836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:20.001610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:33867 (local address 127.18.134.2:40753)
0619 02:55:20.002183 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:20.002206 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:20.002393 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:20.002774 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:20.002778 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:20.002800 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 02:55:20.003140 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:20.003151 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:20.004218 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:20.004223 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:20.006269 (+ 2046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:20.006283 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:20.008057 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:20.008063 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:20.008234 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:20.008265 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 02:55:20.008341 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 02:55:20.008459 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":883,"threads_started":1}
I20260619 02:55:20.008783 6825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:20.001753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.134.2:40753 (local address 127.18.134.1:33867)
0619 02:55:20.001925 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:20.001930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:20.002284 (+ 354us) server_negotiation.cc:407] Connection header received
0619 02:55:20.002535 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:20.002540 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:20.002673 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:20.002746 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 02:55:20.003279 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:20.004058 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:20.006482 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:20.007715 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:20.007825 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:20.007885 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:20.008390 (+ 505us) server_negotiation.cc:299] Negotiation successful
0619 02:55:20.008513 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 02:55:20.009128 6763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "619716c0cd0146759c863ad3e762800b" candidate_uuid: "72f71fef589a46be900e77e2928c1498" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c2f0374e7424fe3ba993ae52a7d9d8d" is_pre_election: true
I20260619 02:55:20.009559 6763 raft_consensus.cc:2454] T 619716c0cd0146759c863ad3e762800b P 0c2f0374e7424fe3ba993ae52a7d9d8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72f71fef589a46be900e77e2928c1498 in term 0.
I20260619 02:55:20.010133 6568 leader_election.cc:302] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c2f0374e7424fe3ba993ae52a7d9d8d, 72f71fef589a46be900e77e2928c1498; no voters:
I20260619 02:55:20.010525 6834 raft_consensus.cc:2781] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 02:55:20.010697 6834 raft_consensus.cc:482] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 02:55:20.010797 6834 raft_consensus.cc:3037] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:20.010766 6503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "619716c0cd0146759c863ad3e762800b" candidate_uuid: "72f71fef589a46be900e77e2928c1498" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96569cbe15ee461ea960a8ba648d2b19" is_pre_election: true
I20260619 02:55:20.011264 6503 raft_consensus.cc:2454] T 619716c0cd0146759c863ad3e762800b P 96569cbe15ee461ea960a8ba648d2b19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72f71fef589a46be900e77e2928c1498 in term 0.
I20260619 02:55:20.013926 6834 raft_consensus.cc:504] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f71fef589a46be900e77e2928c1498" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 34107 } } peers { permanent_uuid: "0c2f0374e7424fe3ba993ae52a7d9d8d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 45569 } } peers { permanent_uuid: "96569cbe15ee461ea960a8ba648d2b19" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33867 } }
I20260619 02:55:20.014555 6834 leader_election.cc:288] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498 [CANDIDATE]: Term 1 election: Requested vote from peers 0c2f0374e7424fe3ba993ae52a7d9d8d (127.18.134.3:45569), 96569cbe15ee461ea960a8ba648d2b19 (127.18.134.1:33867)
I20260619 02:55:20.015017 6503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "619716c0cd0146759c863ad3e762800b" candidate_uuid: "72f71fef589a46be900e77e2928c1498" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96569cbe15ee461ea960a8ba648d2b19"
I20260619 02:55:20.015031 6763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "619716c0cd0146759c863ad3e762800b" candidate_uuid: "72f71fef589a46be900e77e2928c1498" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c2f0374e7424fe3ba993ae52a7d9d8d"
I20260619 02:55:20.015203 6503 raft_consensus.cc:3037] T 619716c0cd0146759c863ad3e762800b P 96569cbe15ee461ea960a8ba648d2b19 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:20.015218 6763 raft_consensus.cc:3037] T 619716c0cd0146759c863ad3e762800b P 0c2f0374e7424fe3ba993ae52a7d9d8d [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:20.017666 6503 raft_consensus.cc:2454] T 619716c0cd0146759c863ad3e762800b P 96569cbe15ee461ea960a8ba648d2b19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72f71fef589a46be900e77e2928c1498 in term 1.
I20260619 02:55:20.018010 6568 leader_election.cc:302] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72f71fef589a46be900e77e2928c1498, 96569cbe15ee461ea960a8ba648d2b19; no voters:
I20260619 02:55:20.018134 6763 raft_consensus.cc:2454] T 619716c0cd0146759c863ad3e762800b P 0c2f0374e7424fe3ba993ae52a7d9d8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72f71fef589a46be900e77e2928c1498 in term 1.
I20260619 02:55:20.018286 6834 raft_consensus.cc:2781] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:20.018810 6834 raft_consensus.cc:686] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498 [term 1 LEADER]: Becoming Leader. State: Replica: 72f71fef589a46be900e77e2928c1498, State: Running, Role: LEADER
I20260619 02:55:20.019285 6834 consensus_queue.cc:237] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f71fef589a46be900e77e2928c1498" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 34107 } } peers { permanent_uuid: "0c2f0374e7424fe3ba993ae52a7d9d8d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 45569 } } peers { permanent_uuid: "96569cbe15ee461ea960a8ba648d2b19" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33867 } }
I20260619 02:55:20.024020 6361 catalog_manager.cc:5495] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72f71fef589a46be900e77e2928c1498 (127.18.134.2). New cstate: current_term: 1 leader_uuid: "72f71fef589a46be900e77e2928c1498" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72f71fef589a46be900e77e2928c1498" member_type: VOTER last_known_addr { host: "127.18.134.2" port: 34107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c2f0374e7424fe3ba993ae52a7d9d8d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 45569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96569cbe15ee461ea960a8ba648d2b19" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33867 } health_report { overall_health: UNKNOWN } } }
I20260619 02:55:20.044322 6822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:20.038961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52388 (local address 127.18.134.2:34107)
0619 02:55:20.039120 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:20.039124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:20.039160 (+ 36us) server_negotiation.cc:407] Connection header received
0619 02:55:20.039347 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:20.039351 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:20.039434 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:20.039562 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:55:20.039959 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:20.040693 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:20.042164 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:20.042607 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:20.042778 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:20.043314 (+ 536us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:55:20.043816 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:55:20.043929 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:20.044064 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 02:55:20.044132 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 02:55:20.052304 6763 raft_consensus.cc:1261] T 619716c0cd0146759c863ad3e762800b P 0c2f0374e7424fe3ba993ae52a7d9d8d [term 1 FOLLOWER]: Refusing update from remote peer 72f71fef589a46be900e77e2928c1498: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:55:20.052304 6503 raft_consensus.cc:1261] T 619716c0cd0146759c863ad3e762800b P 96569cbe15ee461ea960a8ba648d2b19 [term 1 FOLLOWER]: Refusing update from remote peer 72f71fef589a46be900e77e2928c1498: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:55:20.053421 6837 consensus_queue.cc:1035] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96569cbe15ee461ea960a8ba648d2b19" member_type: VOTER last_known_addr { host: "127.18.134.1" port: 33867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:55:20.053793 6834 consensus_queue.cc:1035] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c2f0374e7424fe3ba993ae52a7d9d8d" member_type: VOTER last_known_addr { host: "127.18.134.3" port: 45569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:55:20.075825 6842 mvcc.cc:205] Tried to move back new op lower bound from 7298407301324431360 to 7298407301202898944. Current Snapshot: MvccSnapshot[applied={T|T < 7298407301324431360}]
I20260619 02:55:20.090840 6362 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:55:20.091138 6362 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 02:55:20.095274 6362 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f95f051ea5a14afc8c3074ed214f0612: Sending DeleteTablet for 3 replicas of tablet 619716c0cd0146759c863ad3e762800b
I20260619 02:55:20.096447 6613 tablet_service.cc:1514] Processing DeleteTablet for tablet 619716c0cd0146759c863ad3e762800b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:55:20 UTC) from {username='slave'} at 127.0.0.1:52374
I20260619 02:55:20.096668 6743 tablet_service.cc:1514] Processing DeleteTablet for tablet 619716c0cd0146759c863ad3e762800b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:55:20 UTC) from {username='slave'} at 127.0.0.1:58976
I20260619 02:55:20.097182 6483 tablet_service.cc:1514] Processing DeleteTablet for tablet 619716c0cd0146759c863ad3e762800b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 02:55:20 UTC) from {username='slave'} at 127.0.0.1:55146
I20260619 02:55:20.097347 6849 tablet_replica.cc:330] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498: stopping tablet replica
I20260619 02:55:20.097520 6850 tablet_replica.cc:330] T 619716c0cd0146759c863ad3e762800b P 0c2f0374e7424fe3ba993ae52a7d9d8d: stopping tablet replica
I20260619 02:55:20.097811 6849 raft_consensus.cc:2229] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498 [term 1 LEADER]: Raft consensus shutting down.
I20260619 02:55:20.097896 6850 raft_consensus.cc:2229] T 619716c0cd0146759c863ad3e762800b P 0c2f0374e7424fe3ba993ae52a7d9d8d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:55:20.098361 6850 raft_consensus.cc:2258] T 619716c0cd0146759c863ad3e762800b P 0c2f0374e7424fe3ba993ae52a7d9d8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:55:20.098619 6849 raft_consensus.cc:2258] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:55:20.099560 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 6421
I20260619 02:55:20.100555 6850 ts_tablet_manager.cc:1905] T 619716c0cd0146759c863ad3e762800b P 0c2f0374e7424fe3ba993ae52a7d9d8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:55:20.100814 6849 ts_tablet_manager.cc:1905] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:55:20.107313 6850 ts_tablet_manager.cc:1918] T 619716c0cd0146759c863ad3e762800b P 0c2f0374e7424fe3ba993ae52a7d9d8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:55:20.107524 6850 log.cc:1196] T 619716c0cd0146759c863ad3e762800b P 0c2f0374e7424fe3ba993ae52a7d9d8d: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-2/wal/wals/619716c0cd0146759c863ad3e762800b
I20260619 02:55:20.108067 6850 ts_tablet_manager.cc:1939] T 619716c0cd0146759c863ad3e762800b P 0c2f0374e7424fe3ba993ae52a7d9d8d: Deleting consensus metadata
I20260619 02:55:20.108361 6849 ts_tablet_manager.cc:1918] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 02:55:20.108582 6849 log.cc:1196] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498: Deleting WAL directory at /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781837410515220-18968-0/minicluster-data/ts-1/wal/wals/619716c0cd0146759c863ad3e762800b
I20260619 02:55:20.109185 6849 ts_tablet_manager.cc:1939] T 619716c0cd0146759c863ad3e762800b P 72f71fef589a46be900e77e2928c1498: Deleting consensus metadata
I20260619 02:55:20.110086 6348 catalog_manager.cc:4841] TS 0c2f0374e7424fe3ba993ae52a7d9d8d (127.18.134.3:45569): tablet 619716c0cd0146759c863ad3e762800b (table test-table [id=b75ffd4cd826410ca2f253f96834bd76]) successfully deleted
I20260619 02:55:20.110952 6349 catalog_manager.cc:4841] TS 72f71fef589a46be900e77e2928c1498 (127.18.134.2:34107): tablet 619716c0cd0146759c863ad3e762800b (table test-table [id=b75ffd4cd826410ca2f253f96834bd76]) successfully deleted
I20260619 02:55:20.111828 6851 tablet_replica.cc:330] T 619716c0cd0146759c863ad3e762800b P 96569cbe15ee461ea960a8ba648d2b19: stopping tablet replica
I20260619 02:55:20.112298 6851 raft_consensus.cc:2229] T 619716c0cd0146759c863ad3e762800b P 96569cbe15ee461ea960a8ba648d2b19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 02:55:20.112851 6851 raft_consensus.cc:2258] T 619716c0cd0146759c863ad3e762800b P 96569cbe15ee461ea960a8ba648d2b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 02:55:20.114744 6851 ts_tablet_manager.cc:1905] T 619716c0cd0146759c863ad3e762800b P 96569cbe15ee461ea960a8ba648d2b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 02:55:20.118815 6825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:20.101710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55152 (local address 127.18.134.1:33867)
0619 02:55:20.101893 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:20.101900 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:20.111169 (+ 9269us) server_negotiation.cc:407] Connection header received
0619 02:55:20.111384 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:20.111390 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:20.111528 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:20.111664 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:20.112157 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:20.113124 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:20.114998 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:20.115641 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:20.115779 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:20.117071 (+ 1292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:20.117085 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:20.117102 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:20.117881 (+ 779us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:20.117942 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:20.117979 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:20.118100 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:20.118233 (+ 133us) server_negotiation.cc:299] Negotiation successful
0619 02:55:20.118344 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 02:55:20.119665 6543 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:20.207890 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 6421
I20260619 02:55:20.222553 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 6551
W20260619 02:55:20.222675 6348 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.134.1:33867 (error 108)
I20260619 02:55:20.225857 6824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:20.225386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:33867 (local address 127.0.0.1:55160)
0619 02:55:20.225563 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:20.225681 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260619 02:55:20.226114 6348 catalog_manager.cc:4584] TS 96569cbe15ee461ea960a8ba648d2b19 (127.18.134.1:33867): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 619716c0cd0146759c863ad3e762800b: Network error: Client connection negotiation failed: client connection to 127.18.134.1:33867: connect: Connection refused (error 111)
I20260619 02:55:20.230406 6822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:20.223708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52400 (local address 127.18.134.2:34107)
0619 02:55:20.223906 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:20.223912 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:20.223932 (+ 20us) server_negotiation.cc:407] Connection header received
0619 02:55:20.223995 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:20.224000 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:20.224117 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:20.224231 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:20.224736 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:20.225719 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:20.227501 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:20.227919 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:20.228047 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:20.229066 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:20.229080 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:20.229094 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:20.229676 (+ 582us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:20.229721 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:20.229747 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:20.229838 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:20.229993 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 02:55:20.230091 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 02:55:20.231200 6673 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:20.279943 6824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:20.279356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:33867 (local address 127.0.0.1:55172)
0619 02:55:20.279624 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:20.279742 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260619 02:55:20.281138 6824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:20.280742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:33867 (local address 127.0.0.1:55186)
0619 02:55:20.280892 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:20.280979 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 02:55:20.341704 6824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:20.341158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:33867 (local address 127.0.0.1:55200)
0619 02:55:20.341377 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:20.341492 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260619 02:55:20.342065 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 6551
I20260619 02:55:20.343194 6824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:20.342854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:33867 (local address 127.0.0.1:55202)
0619 02:55:20.343007 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:20.343077 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 02:55:20.356685 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 6681
I20260619 02:55:20.363291 6823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:20.357484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58978 (local address 127.18.134.3:45569)
0619 02:55:20.357653 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:20.357657 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:20.357677 (+ 20us) server_negotiation.cc:407] Connection header received
0619 02:55:20.357738 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:20.357742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:20.357827 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:20.357962 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:20.358370 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:20.359262 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:20.360544 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:20.360952 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:20.361062 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:20.361967 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:20.361974 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:20.361989 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:20.362599 (+ 610us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:20.362702 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:20.362726 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:20.362814 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:20.362932 (+ 118us) server_negotiation.cc:299] Negotiation successful
0619 02:55:20.363019 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 02:55:20.363947 6803 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:20.423867 6824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:20.423393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:33867 (local address 127.0.0.1:55216)
0619 02:55:20.423591 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:20.423690 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260619 02:55:20.424921 6824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:20.424517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.1:33867 (local address 127.0.0.1:55220)
0619 02:55:20.424716 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:20.424788 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.1:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260619 02:55:20.450362 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 6681
I20260619 02:55:20.463403 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 6331
I20260619 02:55:20.464483 6392 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:20.571096 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 6331
2026-06-19T02:55:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2566 ms)
[----------] 6 tests from EncryptionPolicyTest (20207 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:55:20 dist-test-slave-9mr0 krb5kdc[6860](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:55:20 dist-test-slave-9mr0 krb5kdc[6860](info): set up 2 sockets
Jun 19 02:55:20 dist-test-slave-9mr0 krb5kdc[6860](info): commencing operation
krb5kdc: starting...
W20260619 02:55:21.590998 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.984s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:55:21 dist-test-slave-9mr0 krb5kdc[6860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:55:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:55:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:55:21.825762 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46441
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:22.144492 6873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:22.144798 6873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:22.144945 6873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:22.152168 6873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:22.152294 6873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:22.152372 6873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:22.152455 6873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:22.152519 6873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:22.162309 6873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46441
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:22.163625 6873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:22.164844 6873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:22.176133 6878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:22.176060 6879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:22.176084 6881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:22.177245 6873 server_base.cc:1034] running on GCE node
I20260619 02:55:22.178030 6873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:22.179455 6873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:22.180665 6873 hybrid_clock.cc:648] HybridClock initialized: now 1781837722180611 us; error 46 us; skew 500 ppm
I20260619 02:55:22.181007 6873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:22 dist-test-slave-9mr0 krb5kdc[6860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837722, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:22.184024 6873 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:55:22.185462 6873 webserver.cc:460] Webserver started at http://127.0.0.1:37079/ using document root <none> and password file <none>
I20260619 02:55:22.186105 6873 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:22.186203 6873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:22.186447 6873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:22.189361 6873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "2d9ccbbcb90044b4a2067fdc1e2e3821"
format_stamp: "Formatted at 2026-06-19 02:55:22 on dist-test-slave-9mr0"
server_key: "9eb441991c01ab2d7f2a1ddde8ef0931"
server_key_iv: "ab2c8e1d2c65c00c3eedf762271f00e4"
server_key_version: "encryptionkey@0"
I20260619 02:55:22.190076 6873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "2d9ccbbcb90044b4a2067fdc1e2e3821"
format_stamp: "Formatted at 2026-06-19 02:55:22 on dist-test-slave-9mr0"
server_key: "9eb441991c01ab2d7f2a1ddde8ef0931"
server_key_iv: "ab2c8e1d2c65c00c3eedf762271f00e4"
server_key_version: "encryptionkey@0"
I20260619 02:55:22.195156 6873 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 02:55:22.198674 6888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:22.200089 6873 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:55:22.200246 6873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "2d9ccbbcb90044b4a2067fdc1e2e3821"
format_stamp: "Formatted at 2026-06-19 02:55:22 on dist-test-slave-9mr0"
server_key: "9eb441991c01ab2d7f2a1ddde8ef0931"
server_key_iv: "ab2c8e1d2c65c00c3eedf762271f00e4"
server_key_version: "encryptionkey@0"
I20260619 02:55:22.200416 6873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:22.214882 6873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:22.218851 6873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:22.219086 6873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:22.235800 6873 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46441
I20260619 02:55:22.235823 6939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46441 every 8 connection(s)
I20260619 02:55:22.237097 6873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:55:22.238152 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 6873
I20260619 02:55:22.238346 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:55:22.239002 18968 external_mini_cluster.cc:1428] Setting key b49e6bb3362b8107550037f7c2c5231b
I20260619 02:55:22.241202 6940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 19 02:55:22 dist-test-slave-9mr0 krb5kdc[6860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837721, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 02:55:22.253892 6940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d9ccbbcb90044b4a2067fdc1e2e3821: Bootstrap starting.
I20260619 02:55:22.256992 6940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d9ccbbcb90044b4a2067fdc1e2e3821: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:22.257984 6940 log.cc:824] T 00000000000000000000000000000000 P 2d9ccbbcb90044b4a2067fdc1e2e3821: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:22.260713 6940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d9ccbbcb90044b4a2067fdc1e2e3821: No bootstrap required, opened a new log
I20260619 02:55:22.261482 6943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:22.240974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55065 (local address 127.0.0.1:46441)
0619 02:55:22.241721 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:22.241732 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:22.241772 (+ 40us) server_negotiation.cc:407] Connection header received
0619 02:55:22.243391 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:22.243413 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:22.243845 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:22.244175 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:22.245893 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:22.247326 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:22.248988 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:22.249654 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:22.249800 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:22.254395 (+ 4595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:22.254411 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:22.254424 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:22.254460 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:22.257412 (+ 2952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:22.258212 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:22.258219 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:22.258224 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:22.258381 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:22.258889 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:22.258893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:22.258895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:22.259425 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:22.259701 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:22.260206 (+ 505us) server_negotiation.cc:299] Negotiation successful
0619 02:55:22.260509 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":335,"threads_started":1}
I20260619 02:55:22.265201 18968 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:55:22.268473 6940 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d9ccbbcb90044b4a2067fdc1e2e3821 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9ccbbcb90044b4a2067fdc1e2e3821" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46441 } }
I20260619 02:55:22.268827 6940 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d9ccbbcb90044b4a2067fdc1e2e3821 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:22.268914 6940 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d9ccbbcb90044b4a2067fdc1e2e3821 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d9ccbbcb90044b4a2067fdc1e2e3821, State: Initialized, Role: FOLLOWER
I20260619 02:55:22.269430 6940 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d9ccbbcb90044b4a2067fdc1e2e3821 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9ccbbcb90044b4a2067fdc1e2e3821" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46441 } }
I20260619 02:55:22.269694 6940 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d9ccbbcb90044b4a2067fdc1e2e3821 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:55:22.269840 6940 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d9ccbbcb90044b4a2067fdc1e2e3821 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:55:22.270095 6940 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d9ccbbcb90044b4a2067fdc1e2e3821 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:22.273023 6940 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d9ccbbcb90044b4a2067fdc1e2e3821 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9ccbbcb90044b4a2067fdc1e2e3821" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46441 } }
I20260619 02:55:22.273497 6940 leader_election.cc:302] T 00000000000000000000000000000000 P 2d9ccbbcb90044b4a2067fdc1e2e3821 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d9ccbbcb90044b4a2067fdc1e2e3821; no voters:
I20260619 02:55:22.274073 6940 leader_election.cc:288] T 00000000000000000000000000000000 P 2d9ccbbcb90044b4a2067fdc1e2e3821 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:55:22.274194 6950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d9ccbbcb90044b4a2067fdc1e2e3821 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:22.276376 6940 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d9ccbbcb90044b4a2067fdc1e2e3821 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:55:22.281064 6950 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d9ccbbcb90044b4a2067fdc1e2e3821 [term 1 LEADER]: Becoming Leader. State: Replica: 2d9ccbbcb90044b4a2067fdc1e2e3821, State: Running, Role: LEADER
I20260619 02:55:22.281733 6950 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d9ccbbcb90044b4a2067fdc1e2e3821 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9ccbbcb90044b4a2067fdc1e2e3821" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46441 } }
I20260619 02:55:22.283589 6943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:22.269774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33065 (local address 127.0.0.1:46441)
0619 02:55:22.269972 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:22.269979 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:22.270353 (+ 374us) server_negotiation.cc:407] Connection header received
0619 02:55:22.270526 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:22.270530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:22.270587 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:22.270700 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:22.272413 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:22.273183 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:22.274595 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:22.275291 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:22.275366 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:22.277947 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:22.277953 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:22.277958 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:22.278000 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:22.280772 (+ 2772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:22.281516 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:22.281520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:22.281522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:22.281661 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:22.282131 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:22.282135 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:22.282137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:22.282415 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:22.282699 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:22.283059 (+ 360us) server_negotiation.cc:299] Negotiation successful
0619 02:55:22.283217 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 02:55:22.288201 6951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d9ccbbcb90044b4a2067fdc1e2e3821 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d9ccbbcb90044b4a2067fdc1e2e3821" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9ccbbcb90044b4a2067fdc1e2e3821" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46441 } } }
I20260619 02:55:22.288648 6951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d9ccbbcb90044b4a2067fdc1e2e3821 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:22.289184 6952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d9ccbbcb90044b4a2067fdc1e2e3821 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d9ccbbcb90044b4a2067fdc1e2e3821. Latest consensus state: current_term: 1 leader_uuid: "2d9ccbbcb90044b4a2067fdc1e2e3821" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9ccbbcb90044b4a2067fdc1e2e3821" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46441 } } }
I20260619 02:55:22.289513 6952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d9ccbbcb90044b4a2067fdc1e2e3821 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:22.290947 6962 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:55:22.296077 6962 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:55:22.304617 6962 catalog_manager.cc:1347] Generated new cluster ID: 9bc071fc09d547b0882a71e5b263b8e7
I20260619 02:55:22.304718 6962 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:55:22.321144 6962 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:55:22.322485 6962 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:55:22.332870 6962 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d9ccbbcb90044b4a2067fdc1e2e3821: Generated new TSK 0
I20260619 02:55:22.333698 6962 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:55:22.347955 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 6873
I20260619 02:55:22.349251 6935 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:22.439914 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 6873
W20260619 02:55:22.453815 6858 connection.cc:535] client connection to 127.0.0.1:46441 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46441): Connection reset by peer (error 104)
2026-06-19T02:55:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1865 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:55:22 dist-test-slave-9mr0 krb5kdc[6972](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:55:22 dist-test-slave-9mr0 krb5kdc[6972](info): set up 2 sockets
Jun 19 02:55:22 dist-test-slave-9mr0 krb5kdc[6972](info): commencing operation
krb5kdc: starting...
W20260619 02:55:23.442801 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.971s user 0.006s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:55:23 dist-test-slave-9mr0 krb5kdc[6972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:55:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:55:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:55:23.678154 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45871
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:23.995729 6985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:23.996039 6985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:23.996162 6985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:24.003822 6985 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:24.003948 6985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:24.004014 6985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:24.004096 6985 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:24.004163 6985 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:24.014144 6985 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45871
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:24.015514 6985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:24.016819 6985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:24.028311 6990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:24.028285 6991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:24.028239 6993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:24.029474 6985 server_base.cc:1034] running on GCE node
I20260619 02:55:24.030191 6985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:24.031603 6985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:24.032797 6985 hybrid_clock.cc:648] HybridClock initialized: now 1781837724032753 us; error 41 us; skew 500 ppm
I20260619 02:55:24.033118 6985 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:24 dist-test-slave-9mr0 krb5kdc[6972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837724, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:24.036108 6985 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:55:24.037516 6985 webserver.cc:460] Webserver started at http://127.0.0.1:33131/ using document root <none> and password file <none>
I20260619 02:55:24.038130 6985 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:24.038228 6985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:24.038467 6985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:24.041345 6985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "850f93af399146d18bf34a0db597b282"
format_stamp: "Formatted at 2026-06-19 02:55:24 on dist-test-slave-9mr0"
server_key: "b166730519784ef9d193fd390f707482"
server_key_iv: "e293f537d1a99ffe0570371a21694e2f"
server_key_version: "encryptionkey@0"
I20260619 02:55:24.042063 6985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "850f93af399146d18bf34a0db597b282"
format_stamp: "Formatted at 2026-06-19 02:55:24 on dist-test-slave-9mr0"
server_key: "b166730519784ef9d193fd390f707482"
server_key_iv: "e293f537d1a99ffe0570371a21694e2f"
server_key_version: "encryptionkey@0"
I20260619 02:55:24.046973 6985 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260619 02:55:24.050345 7000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:24.051640 6985 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:55:24.051775 6985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "850f93af399146d18bf34a0db597b282"
format_stamp: "Formatted at 2026-06-19 02:55:24 on dist-test-slave-9mr0"
server_key: "b166730519784ef9d193fd390f707482"
server_key_iv: "e293f537d1a99ffe0570371a21694e2f"
server_key_version: "encryptionkey@0"
I20260619 02:55:24.051931 6985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:24.064824 6985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:24.069113 6985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:24.069396 6985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:24.088372 6985 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45871
I20260619 02:55:24.088392 7051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45871 every 8 connection(s)
I20260619 02:55:24.089815 6985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:55:24.090818 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 6985
I20260619 02:55:24.091005 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:55:24.091529 18968 external_mini_cluster.cc:1428] Setting key 9b4c592f335264d3fbb9d713255a5ea8
I20260619 02:55:24.094384 7052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:24.103914 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:24.093274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52907 (local address 127.0.0.1:45871)
0619 02:55:24.094018 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:24.094029 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:24.094070 (+ 41us) server_negotiation.cc:407] Connection header received
0619 02:55:24.095778 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:24.095800 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:24.096180 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:24.096515 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:24.097079 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:24.098392 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:24.099724 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:24.100379 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:24.100547 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:24.101709 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:24.101716 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:24.101732 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:24.102307 (+ 575us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:24.102387 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:24.102412 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:24.102568 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:24.102857 (+ 289us) server_negotiation.cc:299] Negotiation successful
0619 02:55:24.103014 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":296,"threads_started":1}
I20260619 02:55:24.104996 7052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 850f93af399146d18bf34a0db597b282: Bootstrap starting.
I20260619 02:55:24.107883 18968 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:55:24.108314 7052 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 850f93af399146d18bf34a0db597b282: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:24.109261 7052 log.cc:824] T 00000000000000000000000000000000 P 850f93af399146d18bf34a0db597b282: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:24.112435 7052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 850f93af399146d18bf34a0db597b282: No bootstrap required, opened a new log
I20260619 02:55:24.118100 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:24.112052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42171 (local address 127.0.0.1:45871)
0619 02:55:24.112207 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:24.112211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:24.112663 (+ 452us) server_negotiation.cc:407] Connection header received
0619 02:55:24.112849 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:24.112853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:24.112914 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:24.113000 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:24.113525 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:24.114303 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:24.115642 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:24.116073 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:24.116180 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:24.117397 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:24.117402 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:24.117406 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:24.117423 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:24.117465 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:24.117483 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:24.117590 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:24.117781 (+ 191us) server_negotiation.cc:299] Negotiation successful
0619 02:55:24.117843 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 02:55:24.120554 7052 raft_consensus.cc:348] T 00000000000000000000000000000000 P 850f93af399146d18bf34a0db597b282 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "850f93af399146d18bf34a0db597b282" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45871 } }
I20260619 02:55:24.120919 7052 raft_consensus.cc:374] T 00000000000000000000000000000000 P 850f93af399146d18bf34a0db597b282 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:24.121014 7052 raft_consensus.cc:729] T 00000000000000000000000000000000 P 850f93af399146d18bf34a0db597b282 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 850f93af399146d18bf34a0db597b282, State: Initialized, Role: FOLLOWER
I20260619 02:55:24.121569 7052 consensus_queue.cc:260] T 00000000000000000000000000000000 P 850f93af399146d18bf34a0db597b282 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "850f93af399146d18bf34a0db597b282" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45871 } }
I20260619 02:55:24.121773 7052 raft_consensus.cc:388] T 00000000000000000000000000000000 P 850f93af399146d18bf34a0db597b282 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:55:24.121892 7052 raft_consensus.cc:482] T 00000000000000000000000000000000 P 850f93af399146d18bf34a0db597b282 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:55:24.122058 7052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 850f93af399146d18bf34a0db597b282 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:24.124301 7052 raft_consensus.cc:504] T 00000000000000000000000000000000 P 850f93af399146d18bf34a0db597b282 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "850f93af399146d18bf34a0db597b282" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45871 } }
I20260619 02:55:24.124894 7052 leader_election.cc:302] T 00000000000000000000000000000000 P 850f93af399146d18bf34a0db597b282 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 850f93af399146d18bf34a0db597b282; no voters:
I20260619 02:55:24.125427 7052 leader_election.cc:288] T 00000000000000000000000000000000 P 850f93af399146d18bf34a0db597b282 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:55:24.125553 7062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 850f93af399146d18bf34a0db597b282 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:24.126245 7062 raft_consensus.cc:686] T 00000000000000000000000000000000 P 850f93af399146d18bf34a0db597b282 [term 1 LEADER]: Becoming Leader. State: Replica: 850f93af399146d18bf34a0db597b282, State: Running, Role: LEADER
I20260619 02:55:24.126787 7062 consensus_queue.cc:237] T 00000000000000000000000000000000 P 850f93af399146d18bf34a0db597b282 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "850f93af399146d18bf34a0db597b282" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45871 } }
I20260619 02:55:24.127563 7052 sys_catalog.cc:564] T 00000000000000000000000000000000 P 850f93af399146d18bf34a0db597b282 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:55:24.129371 7063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 850f93af399146d18bf34a0db597b282 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "850f93af399146d18bf34a0db597b282" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "850f93af399146d18bf34a0db597b282" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45871 } } }
I20260619 02:55:24.129707 7063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 850f93af399146d18bf34a0db597b282 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:24.131330 7067 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:55:24.131307 7064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 850f93af399146d18bf34a0db597b282 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 850f93af399146d18bf34a0db597b282. Latest consensus state: current_term: 1 leader_uuid: "850f93af399146d18bf34a0db597b282" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "850f93af399146d18bf34a0db597b282" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45871 } } }
I20260619 02:55:24.131597 7064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 850f93af399146d18bf34a0db597b282 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:24.136646 7067 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:55:24.145742 7067 catalog_manager.cc:1347] Generated new cluster ID: 54bc48657c694eceb3ebea4fd1afc756
I20260619 02:55:24.145867 7067 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:55:24.174254 7067 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:55:24.175586 7067 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:55:24.187801 7067 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 850f93af399146d18bf34a0db597b282: Generated new TSK 0
I20260619 02:55:24.188771 7067 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:55:24.207810 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 6985
W20260619 02:55:24.208993 7047 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:52907
W20260619 02:55:24.209746 18968 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 6985: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 02:55:24.209930 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 6985
W20260619 02:55:24.223206 6970 connection.cc:535] client connection to 127.0.0.1:45871 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45871): Connection reset by peer (error 104)
2026-06-19T02:55:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1769 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:55:24 dist-test-slave-9mr0 krb5kdc[7083](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:55:24 dist-test-slave-9mr0 krb5kdc[7083](info): set up 2 sockets
Jun 19 02:55:24 dist-test-slave-9mr0 krb5kdc[7083](info): commencing operation
krb5kdc: starting...
W20260619 02:55:25.220245 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.979s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:55:25 dist-test-slave-9mr0 krb5kdc[7083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:55:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:55:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:55:25.458125 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43501
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43501
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:25.775789 7096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:25.776104 7096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:25.776238 7096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:25.783532 7096 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:25.783658 7096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:25.783736 7096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:25.783814 7096 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:25.783872 7096 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:25.793605 7096 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43501
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:25.794914 7096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:25.796207 7096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:25.807366 7101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:25.807565 7104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:25.807605 7102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:25.808408 7096 server_base.cc:1034] running on GCE node
I20260619 02:55:25.809204 7096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:25.810571 7096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:25.811813 7096 hybrid_clock.cc:648] HybridClock initialized: now 1781837725811762 us; error 43 us; skew 500 ppm
I20260619 02:55:25.812150 7096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:25 dist-test-slave-9mr0 krb5kdc[7083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837725, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:25.821058 7096 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:55:25.822575 7096 webserver.cc:460] Webserver started at http://127.0.0.1:43213/ using document root <none> and password file <none>
I20260619 02:55:25.823252 7096 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:25.823376 7096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:25.823669 7096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:25.826551 7096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "d1bc5b4eef404bf9a76d31273be6385c"
format_stamp: "Formatted at 2026-06-19 02:55:25 on dist-test-slave-9mr0"
server_key: "bf9222a0c6ee88e5972528da25a49d16"
server_key_iv: "b72ced27163b4f14706f5472f4bd388a"
server_key_version: "encryptionkey@0"
I20260619 02:55:25.827323 7096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "d1bc5b4eef404bf9a76d31273be6385c"
format_stamp: "Formatted at 2026-06-19 02:55:25 on dist-test-slave-9mr0"
server_key: "bf9222a0c6ee88e5972528da25a49d16"
server_key_iv: "b72ced27163b4f14706f5472f4bd388a"
server_key_version: "encryptionkey@0"
I20260619 02:55:25.832458 7096 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:55:25.835920 7111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:25.837226 7096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:55:25.837397 7096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "d1bc5b4eef404bf9a76d31273be6385c"
format_stamp: "Formatted at 2026-06-19 02:55:25 on dist-test-slave-9mr0"
server_key: "bf9222a0c6ee88e5972528da25a49d16"
server_key_iv: "b72ced27163b4f14706f5472f4bd388a"
server_key_version: "encryptionkey@0"
I20260619 02:55:25.837584 7096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:25.843133 7096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:25.847065 7096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:25.847317 7096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:25.864463 7096 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43501
I20260619 02:55:25.864504 7162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43501 every 8 connection(s)
I20260619 02:55:25.865866 7096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:55:25.869549 7163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:25.870134 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 7096
I20260619 02:55:25.870276 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:55:25.870740 18968 external_mini_cluster.cc:1428] Setting key 95b8088aecc4a2cfbd0f02f00f8eb73c
I20260619 02:55:25.878343 7166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:25.872381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46343 (local address 127.0.0.1:43501)
0619 02:55:25.873113 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:25.873124 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:25.873165 (+ 41us) server_negotiation.cc:407] Connection header received
0619 02:55:25.875215 (+ 2050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:25.875240 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:25.875567 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:25.875888 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:25.876934 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:25.876940 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:25.876952 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:25.876965 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:25.877021 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:25.877044 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:25.877195 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:25.877297 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 02:55:25.877448 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":327,"threads_started":1}
I20260619 02:55:25.881953 18968 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:55:25.882335 7163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1bc5b4eef404bf9a76d31273be6385c: Bootstrap starting.
I20260619 02:55:25.885258 7163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1bc5b4eef404bf9a76d31273be6385c: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:25.886354 7163 log.cc:824] T 00000000000000000000000000000000 P d1bc5b4eef404bf9a76d31273be6385c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:25.888486 7166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:25.886066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57755 (local address 127.0.0.1:43501)
0619 02:55:25.886247 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:25.886253 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:25.886561 (+ 308us) server_negotiation.cc:407] Connection header received
0619 02:55:25.886753 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:25.886757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:25.886802 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:25.886931 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:25.887868 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:25.887874 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:25.887878 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:25.887916 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:25.887953 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:25.887968 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:25.888067 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:25.888177 (+ 110us) server_negotiation.cc:299] Negotiation successful
0619 02:55:25.888235 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:55:25.889261 7163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1bc5b4eef404bf9a76d31273be6385c: No bootstrap required, opened a new log
I20260619 02:55:25.896523 7163 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1bc5b4eef404bf9a76d31273be6385c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1bc5b4eef404bf9a76d31273be6385c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43501 } }
I20260619 02:55:25.896907 7163 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1bc5b4eef404bf9a76d31273be6385c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:25.897020 7163 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1bc5b4eef404bf9a76d31273be6385c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1bc5b4eef404bf9a76d31273be6385c, State: Initialized, Role: FOLLOWER
I20260619 02:55:25.897553 7163 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1bc5b4eef404bf9a76d31273be6385c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1bc5b4eef404bf9a76d31273be6385c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43501 } }
I20260619 02:55:25.897771 7163 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1bc5b4eef404bf9a76d31273be6385c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:55:25.897894 7163 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1bc5b4eef404bf9a76d31273be6385c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:55:25.898051 7163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1bc5b4eef404bf9a76d31273be6385c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:25.900435 7163 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1bc5b4eef404bf9a76d31273be6385c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1bc5b4eef404bf9a76d31273be6385c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43501 } }
I20260619 02:55:25.900874 7163 leader_election.cc:302] T 00000000000000000000000000000000 P d1bc5b4eef404bf9a76d31273be6385c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1bc5b4eef404bf9a76d31273be6385c; no voters:
I20260619 02:55:25.901409 7163 leader_election.cc:288] T 00000000000000000000000000000000 P d1bc5b4eef404bf9a76d31273be6385c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:55:25.901544 7173 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1bc5b4eef404bf9a76d31273be6385c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:25.902204 7173 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1bc5b4eef404bf9a76d31273be6385c [term 1 LEADER]: Becoming Leader. State: Replica: d1bc5b4eef404bf9a76d31273be6385c, State: Running, Role: LEADER
I20260619 02:55:25.902760 7173 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1bc5b4eef404bf9a76d31273be6385c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1bc5b4eef404bf9a76d31273be6385c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43501 } }
I20260619 02:55:25.903343 7163 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1bc5b4eef404bf9a76d31273be6385c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:55:25.905174 7174 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1bc5b4eef404bf9a76d31273be6385c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1bc5b4eef404bf9a76d31273be6385c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1bc5b4eef404bf9a76d31273be6385c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43501 } } }
I20260619 02:55:25.905616 7174 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1bc5b4eef404bf9a76d31273be6385c [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:25.905562 7175 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1bc5b4eef404bf9a76d31273be6385c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1bc5b4eef404bf9a76d31273be6385c. Latest consensus state: current_term: 1 leader_uuid: "d1bc5b4eef404bf9a76d31273be6385c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1bc5b4eef404bf9a76d31273be6385c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43501 } } }
I20260619 02:55:25.905898 7175 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1bc5b4eef404bf9a76d31273be6385c [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:25.907826 7178 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:55:25.912859 7178 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:55:25.921666 7178 catalog_manager.cc:1347] Generated new cluster ID: 04b92374d97541f788dc9a8421809483
I20260619 02:55:25.921797 7178 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:55:25.988482 7178 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:55:25.988991 7178 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:55:26.008508 7178 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1bc5b4eef404bf9a76d31273be6385c: Generated new TSK 0
I20260619 02:55:26.009668 7178 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:55:26.028497 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 7096
W20260619 02:55:26.029601 7158 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46343
W20260619 02:55:26.030308 18968 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 7096: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 02:55:26.030471 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 7096
2026-06-19T02:55:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1820 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:55:26 dist-test-slave-9mr0 krb5kdc[7194](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:55:26 dist-test-slave-9mr0 krb5kdc[7194](info): set up 2 sockets
Jun 19 02:55:26 dist-test-slave-9mr0 krb5kdc[7194](info): commencing operation
krb5kdc: starting...
W20260619 02:55:27.042814 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.980s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:55:27 dist-test-slave-9mr0 krb5kdc[7194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:55:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:55:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:55:27.278329 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37647
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:27.588121 7207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:27.588402 7207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:27.588521 7207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:27.595810 7207 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:27.595930 7207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:27.596004 7207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:27.596081 7207 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:27.596149 7207 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:27.606212 7207 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:27.607443 7207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:27.608657 7207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:27.619313 7212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:27.619424 7215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:27.619477 7213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:27.620340 7207 server_base.cc:1034] running on GCE node
I20260619 02:55:27.621037 7207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:27.622386 7207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:27.623608 7207 hybrid_clock.cc:648] HybridClock initialized: now 1781837727623535 us; error 63 us; skew 500 ppm
I20260619 02:55:27.623939 7207 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:27 dist-test-slave-9mr0 krb5kdc[7194](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837727, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:27.626881 7207 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:55:27.628460 7207 webserver.cc:460] Webserver started at http://127.0.0.1:34123/ using document root <none> and password file <none>
I20260619 02:55:27.629086 7207 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:27.629201 7207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:27.629491 7207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:27.632308 7207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "17640ca053d548f7bfafd08db741201c"
format_stamp: "Formatted at 2026-06-19 02:55:27 on dist-test-slave-9mr0"
server_key: "e9bba5ab74267242699acc6a6757c3f7"
server_key_iv: "0878cd84c1378dbbb48c91d45484f786"
server_key_version: "encryptionkey@0"
I20260619 02:55:27.633037 7207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "17640ca053d548f7bfafd08db741201c"
format_stamp: "Formatted at 2026-06-19 02:55:27 on dist-test-slave-9mr0"
server_key: "e9bba5ab74267242699acc6a6757c3f7"
server_key_iv: "0878cd84c1378dbbb48c91d45484f786"
server_key_version: "encryptionkey@0"
I20260619 02:55:27.637820 7207 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 02:55:27.641284 7222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:27.642592 7207 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 02:55:27.642812 7207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "17640ca053d548f7bfafd08db741201c"
format_stamp: "Formatted at 2026-06-19 02:55:27 on dist-test-slave-9mr0"
server_key: "e9bba5ab74267242699acc6a6757c3f7"
server_key_iv: "0878cd84c1378dbbb48c91d45484f786"
server_key_version: "encryptionkey@0"
I20260619 02:55:27.642990 7207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:27.672286 7207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:27.676878 7207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:27.677122 7207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:27.694528 7207 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37647
I20260619 02:55:27.694561 7273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37647 every 8 connection(s)
I20260619 02:55:27.695834 7207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:55:27.699427 7274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:27.700882 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 7207
I20260619 02:55:27.701062 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:55:27.701582 18968 external_mini_cluster.cc:1428] Setting key c3918f815e0c586843b0e6404d7de9dd
I20260619 02:55:27.713447 7274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17640ca053d548f7bfafd08db741201c: Bootstrap starting.
Jun 19 02:55:27 dist-test-slave-9mr0 krb5kdc[7194](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 02:55:27.717365 7274 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17640ca053d548f7bfafd08db741201c: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:27.718459 7274 log.cc:824] T 00000000000000000000000000000000 P 17640ca053d548f7bfafd08db741201c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:27.721769 7274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17640ca053d548f7bfafd08db741201c: No bootstrap required, opened a new log
I20260619 02:55:27.724831 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:27.703352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51425 (local address 127.0.0.1:37647)
0619 02:55:27.704120 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:27.704131 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:27.704172 (+ 41us) server_negotiation.cc:407] Connection header received
0619 02:55:27.705715 (+ 1543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:27.705737 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:27.706151 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:27.706485 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:27.708235 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:27.709527 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:27.711309 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:27.712074 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:27.712231 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:27.717177 (+ 4946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:27.717185 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:27.717199 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:27.717239 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:27.720530 (+ 3291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:27.721376 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:27.721382 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:27.721388 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:27.721549 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:27.721992 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:27.721996 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:27.721998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:27.722529 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:27.722790 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:27.723194 (+ 404us) server_negotiation.cc:299] Negotiation successful
0619 02:55:27.723603 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":348,"threads_started":1}
I20260619 02:55:27.728405 18968 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:55:27.730147 7274 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17640ca053d548f7bfafd08db741201c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17640ca053d548f7bfafd08db741201c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37647 } }
I20260619 02:55:27.730466 7274 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17640ca053d548f7bfafd08db741201c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:27.730559 7274 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17640ca053d548f7bfafd08db741201c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17640ca053d548f7bfafd08db741201c, State: Initialized, Role: FOLLOWER
I20260619 02:55:27.731169 7274 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17640ca053d548f7bfafd08db741201c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17640ca053d548f7bfafd08db741201c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37647 } }
I20260619 02:55:27.731449 7274 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17640ca053d548f7bfafd08db741201c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:55:27.731582 7274 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17640ca053d548f7bfafd08db741201c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:55:27.731738 7274 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17640ca053d548f7bfafd08db741201c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:27.734236 7274 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17640ca053d548f7bfafd08db741201c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17640ca053d548f7bfafd08db741201c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37647 } }
I20260619 02:55:27.734767 7274 leader_election.cc:302] T 00000000000000000000000000000000 P 17640ca053d548f7bfafd08db741201c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17640ca053d548f7bfafd08db741201c; no voters:
I20260619 02:55:27.735363 7274 leader_election.cc:288] T 00000000000000000000000000000000 P 17640ca053d548f7bfafd08db741201c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:55:27.735479 7284 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17640ca053d548f7bfafd08db741201c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:27.737665 7274 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17640ca053d548f7bfafd08db741201c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:55:27.742336 7284 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17640ca053d548f7bfafd08db741201c [term 1 LEADER]: Becoming Leader. State: Replica: 17640ca053d548f7bfafd08db741201c, State: Running, Role: LEADER
I20260619 02:55:27.743023 7284 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17640ca053d548f7bfafd08db741201c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17640ca053d548f7bfafd08db741201c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37647 } }
I20260619 02:55:27.745605 7277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:27.732512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48233 (local address 127.0.0.1:37647)
0619 02:55:27.732663 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:27.732670 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:27.733091 (+ 421us) server_negotiation.cc:407] Connection header received
0619 02:55:27.733250 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:27.733253 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:27.733315 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:27.733400 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:27.734841 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:27.735617 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:27.737099 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:27.737606 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:27.737714 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:27.740031 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:27.740035 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:27.740038 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:27.740068 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:27.742844 (+ 2776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:27.743625 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:27.743631 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:27.743635 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:27.743754 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:27.744121 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:27.744126 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:27.744130 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:27.744476 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:27.744707 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:27.745040 (+ 333us) server_negotiation.cc:299] Negotiation successful
0619 02:55:27.745241 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 02:55:27.751578 7285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17640ca053d548f7bfafd08db741201c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17640ca053d548f7bfafd08db741201c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17640ca053d548f7bfafd08db741201c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37647 } } }
I20260619 02:55:27.752012 7285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17640ca053d548f7bfafd08db741201c [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:27.753175 7286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17640ca053d548f7bfafd08db741201c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17640ca053d548f7bfafd08db741201c. Latest consensus state: current_term: 1 leader_uuid: "17640ca053d548f7bfafd08db741201c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17640ca053d548f7bfafd08db741201c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37647 } } }
I20260619 02:55:27.753476 7286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17640ca053d548f7bfafd08db741201c [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:27.754534 7296 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:55:27.759197 7296 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:55:27.767431 7296 catalog_manager.cc:1347] Generated new cluster ID: f96cf9537cd74dae92c07c8b95d3e1b3
I20260619 02:55:27.767544 7296 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:55:27.789501 7296 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:55:27.790993 7296 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:55:27.798539 7296 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17640ca053d548f7bfafd08db741201c: Generated new TSK 0
I20260619 02:55:27.799340 7296 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:55:27.813544 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 7207
I20260619 02:55:27.814786 7269 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:27.906670 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 7207
2026-06-19T02:55:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1876 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:55:27 dist-test-slave-9mr0 krb5kdc[7306](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:55:27 dist-test-slave-9mr0 krb5kdc[7306](info): set up 2 sockets
Jun 19 02:55:27 dist-test-slave-9mr0 krb5kdc[7306](info): commencing operation
krb5kdc: starting...
W20260619 02:55:28.918835 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.979s user 0.007s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:55:29 dist-test-slave-9mr0 krb5kdc[7306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:55:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:55:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:55:29.155381 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37245
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:29.470537 7319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:29.470896 7319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:29.471042 7319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:29.478236 7319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:29.478361 7319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:29.478440 7319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:29.478533 7319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:29.478590 7319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:29.488734 7319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:29.489934 7319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:29.491192 7319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:29.502389 7325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:29.502415 7327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:29.502537 7324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:29.503531 7319 server_base.cc:1034] running on GCE node
I20260619 02:55:29.504245 7319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:29.505573 7319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:29.506722 7319 hybrid_clock.cc:648] HybridClock initialized: now 1781837729506670 us; error 43 us; skew 500 ppm
I20260619 02:55:29.507055 7319 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:29 dist-test-slave-9mr0 krb5kdc[7306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837729, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:29.510003 7319 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:55:29.511510 7319 webserver.cc:460] Webserver started at http://127.0.0.1:45469/ using document root <none> and password file <none>
I20260619 02:55:29.512157 7319 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:29.512274 7319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:29.512558 7319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:29.515504 7319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "7bb9e2c86f86484ebc0c1f70011a8a46"
format_stamp: "Formatted at 2026-06-19 02:55:29 on dist-test-slave-9mr0"
server_key: "74aacf11b4bf2b717aea0d2ab96187be"
server_key_iv: "41eafc5d7c8a6dc2f38a54e824c802f5"
server_key_version: "encryptionkey@0"
I20260619 02:55:29.516242 7319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "7bb9e2c86f86484ebc0c1f70011a8a46"
format_stamp: "Formatted at 2026-06-19 02:55:29 on dist-test-slave-9mr0"
server_key: "74aacf11b4bf2b717aea0d2ab96187be"
server_key_iv: "41eafc5d7c8a6dc2f38a54e824c802f5"
server_key_version: "encryptionkey@0"
I20260619 02:55:29.521276 7319 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:55:29.524642 7334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:29.525890 7319 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260619 02:55:29.526048 7319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "7bb9e2c86f86484ebc0c1f70011a8a46"
format_stamp: "Formatted at 2026-06-19 02:55:29 on dist-test-slave-9mr0"
server_key: "74aacf11b4bf2b717aea0d2ab96187be"
server_key_iv: "41eafc5d7c8a6dc2f38a54e824c802f5"
server_key_version: "encryptionkey@0"
I20260619 02:55:29.526221 7319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:29.560523 7319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:29.564689 7319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:29.564960 7319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:29.582310 7319 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37245
I20260619 02:55:29.582376 7385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37245 every 8 connection(s)
I20260619 02:55:29.583670 7319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:55:29.587260 7386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:29.588304 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 7319
I20260619 02:55:29.588474 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:55:29.588881 18968 external_mini_cluster.cc:1428] Setting key 5e80e53b9e95015b50c02700934bad94
I20260619 02:55:29.601413 7386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bb9e2c86f86484ebc0c1f70011a8a46: Bootstrap starting.
I20260619 02:55:29.602986 7389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:29.590601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41573 (local address 127.0.0.1:37245)
0619 02:55:29.591407 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:29.591423 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:29.591474 (+ 51us) server_negotiation.cc:407] Connection header received
0619 02:55:29.593485 (+ 2011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:29.593515 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:29.594051 (+ 536us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:29.594495 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:29.595032 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:29.596852 (+ 1820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:29.598227 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:29.599297 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:29.599464 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:29.600533 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:29.600543 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:29.600560 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:29.601247 (+ 687us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:29.601343 (+ 96us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:29.601375 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:29.601498 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:29.601753 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 02:55:29.602002 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":297,"threads_started":1}
I20260619 02:55:29.604611 7386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7bb9e2c86f86484ebc0c1f70011a8a46: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:29.605701 7386 log.cc:824] T 00000000000000000000000000000000 P 7bb9e2c86f86484ebc0c1f70011a8a46: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:29.606362 18968 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:55:29.608235 7386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bb9e2c86f86484ebc0c1f70011a8a46: No bootstrap required, opened a new log
I20260619 02:55:29.616721 7389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:29.610336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36881 (local address 127.0.0.1:37245)
0619 02:55:29.610526 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:29.610533 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:29.610983 (+ 450us) server_negotiation.cc:407] Connection header received
0619 02:55:29.611108 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:29.611112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:29.611172 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:29.611250 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:29.611816 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:29.612542 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:29.614385 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:29.614871 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:29.614989 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:29.616036 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:29.616041 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:29.616045 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:29.616065 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:29.616102 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:29.616119 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:29.616184 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:29.616324 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 02:55:29.616399 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 02:55:29.617354 7386 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7bb9e2c86f86484ebc0c1f70011a8a46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb9e2c86f86484ebc0c1f70011a8a46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37245 } }
I20260619 02:55:29.617702 7386 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7bb9e2c86f86484ebc0c1f70011a8a46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:29.617799 7386 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7bb9e2c86f86484ebc0c1f70011a8a46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bb9e2c86f86484ebc0c1f70011a8a46, State: Initialized, Role: FOLLOWER
I20260619 02:55:29.618330 7386 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7bb9e2c86f86484ebc0c1f70011a8a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb9e2c86f86484ebc0c1f70011a8a46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37245 } }
I20260619 02:55:29.618558 7386 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7bb9e2c86f86484ebc0c1f70011a8a46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:55:29.618696 7386 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7bb9e2c86f86484ebc0c1f70011a8a46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:55:29.618851 7386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7bb9e2c86f86484ebc0c1f70011a8a46 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:29.620945 7386 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7bb9e2c86f86484ebc0c1f70011a8a46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb9e2c86f86484ebc0c1f70011a8a46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37245 } }
I20260619 02:55:29.621474 7386 leader_election.cc:302] T 00000000000000000000000000000000 P 7bb9e2c86f86484ebc0c1f70011a8a46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7bb9e2c86f86484ebc0c1f70011a8a46; no voters:
I20260619 02:55:29.622023 7386 leader_election.cc:288] T 00000000000000000000000000000000 P 7bb9e2c86f86484ebc0c1f70011a8a46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:55:29.622148 7396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7bb9e2c86f86484ebc0c1f70011a8a46 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:29.622781 7396 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7bb9e2c86f86484ebc0c1f70011a8a46 [term 1 LEADER]: Becoming Leader. State: Replica: 7bb9e2c86f86484ebc0c1f70011a8a46, State: Running, Role: LEADER
I20260619 02:55:29.623302 7396 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7bb9e2c86f86484ebc0c1f70011a8a46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb9e2c86f86484ebc0c1f70011a8a46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37245 } }
I20260619 02:55:29.624056 7386 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7bb9e2c86f86484ebc0c1f70011a8a46 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:55:29.626207 7397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bb9e2c86f86484ebc0c1f70011a8a46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7bb9e2c86f86484ebc0c1f70011a8a46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb9e2c86f86484ebc0c1f70011a8a46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37245 } } }
I20260619 02:55:29.626255 7398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bb9e2c86f86484ebc0c1f70011a8a46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7bb9e2c86f86484ebc0c1f70011a8a46. Latest consensus state: current_term: 1 leader_uuid: "7bb9e2c86f86484ebc0c1f70011a8a46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bb9e2c86f86484ebc0c1f70011a8a46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37245 } } }
I20260619 02:55:29.626607 7397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bb9e2c86f86484ebc0c1f70011a8a46 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:29.626621 7398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bb9e2c86f86484ebc0c1f70011a8a46 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:29.631520 7403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:55:29.636843 7403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:55:29.645267 7403 catalog_manager.cc:1347] Generated new cluster ID: c8ae0b9794324b2496d17e5c73a8c7f0
I20260619 02:55:29.645390 7403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:55:29.659394 7403 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:55:29.660727 7403 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:55:29.666535 7403 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7bb9e2c86f86484ebc0c1f70011a8a46: Generated new TSK 0
I20260619 02:55:29.667326 7403 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:55:29.683270 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 7319
W20260619 02:55:29.684466 7381 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41573
W20260619 02:55:29.685242 18968 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 7319: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 02:55:29.685381 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 7319
2026-06-19T02:55:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1777 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:55:29 dist-test-slave-9mr0 krb5kdc[7417](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:55:29 dist-test-slave-9mr0 krb5kdc[7417](info): set up 2 sockets
Jun 19 02:55:29 dist-test-slave-9mr0 krb5kdc[7417](info): commencing operation
krb5kdc: starting...
W20260619 02:55:30.694835 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.978s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:55:30 dist-test-slave-9mr0 krb5kdc[7417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:55:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:55:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:55:30.934046 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33693
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:31.251268 7430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:31.251536 7430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:31.251667 7430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:31.258752 7430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:31.258854 7430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:31.258913 7430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:31.258998 7430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:31.259037 7430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:31.268697 7430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:31.269891 7430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:31.271167 7430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:31.282114 7435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:31.282123 7436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:31.282155 7438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:31.283226 7430 server_base.cc:1034] running on GCE node
I20260619 02:55:31.283910 7430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:31.285211 7430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:31.286401 7430 hybrid_clock.cc:648] HybridClock initialized: now 1781837731286365 us; error 32 us; skew 500 ppm
I20260619 02:55:31.286746 7430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:31 dist-test-slave-9mr0 krb5kdc[7417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837731, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:31.295213 7430 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:55:31.296674 7430 webserver.cc:460] Webserver started at http://127.0.0.1:37403/ using document root <none> and password file <none>
I20260619 02:55:31.297325 7430 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:31.297438 7430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:31.297719 7430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:31.300650 7430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "abb511ece9554e23b6d88e7ea92b9ec5"
format_stamp: "Formatted at 2026-06-19 02:55:31 on dist-test-slave-9mr0"
server_key: "c11ae6f62e1fd69195efdd0d5ed0e1ea"
server_key_iv: "c9472a10da2ad1bf8ca65d6dc6023aa9"
server_key_version: "encryptionkey@0"
I20260619 02:55:31.301398 7430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "abb511ece9554e23b6d88e7ea92b9ec5"
format_stamp: "Formatted at 2026-06-19 02:55:31 on dist-test-slave-9mr0"
server_key: "c11ae6f62e1fd69195efdd0d5ed0e1ea"
server_key_iv: "c9472a10da2ad1bf8ca65d6dc6023aa9"
server_key_version: "encryptionkey@0"
I20260619 02:55:31.306473 7430 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 02:55:31.309824 7445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:31.311122 7430 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:55:31.311267 7430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "abb511ece9554e23b6d88e7ea92b9ec5"
format_stamp: "Formatted at 2026-06-19 02:55:31 on dist-test-slave-9mr0"
server_key: "c11ae6f62e1fd69195efdd0d5ed0e1ea"
server_key_iv: "c9472a10da2ad1bf8ca65d6dc6023aa9"
server_key_version: "encryptionkey@0"
I20260619 02:55:31.311439 7430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:31.317412 7430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:31.321122 7430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:31.321375 7430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:31.338536 7430 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33693
I20260619 02:55:31.338603 7496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33693 every 8 connection(s)
I20260619 02:55:31.339855 7430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:55:31.343432 7497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:31.346426 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 7430
I20260619 02:55:31.346602 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:55:31.347162 18968 external_mini_cluster.cc:1428] Setting key eb30ccdc0435fcbbbfc5f72774facbc0
I20260619 02:55:31.354305 7500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:31.348832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36433 (local address 127.0.0.1:33693)
0619 02:55:31.349566 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:31.349577 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:31.349617 (+ 40us) server_negotiation.cc:407] Connection header received
0619 02:55:31.351244 (+ 1627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:31.351270 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:31.351612 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:31.351919 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:31.352903 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:31.352909 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:31.352920 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:31.352942 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:31.352999 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:31.353018 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:31.353172 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:31.353274 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 02:55:31.353436 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":327,"threads_started":1}
I20260619 02:55:31.354984 7497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abb511ece9554e23b6d88e7ea92b9ec5: Bootstrap starting.
I20260619 02:55:31.357685 18968 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:55:31.358094 7497 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abb511ece9554e23b6d88e7ea92b9ec5: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:31.359300 7497 log.cc:824] T 00000000000000000000000000000000 P abb511ece9554e23b6d88e7ea92b9ec5: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:31.362350 7497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abb511ece9554e23b6d88e7ea92b9ec5: No bootstrap required, opened a new log
I20260619 02:55:31.364668 7500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:31.362411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34495 (local address 127.0.0.1:33693)
0619 02:55:31.362588 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:31.362594 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:31.362986 (+ 392us) server_negotiation.cc:407] Connection header received
0619 02:55:31.363128 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:31.363132 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:31.363179 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:31.363251 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:31.364096 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:31.364100 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:31.364103 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:31.364117 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:31.364159 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:31.364174 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:31.364263 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:31.364353 (+ 90us) server_negotiation.cc:299] Negotiation successful
0619 02:55:31.364399 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 02:55:31.370047 7497 raft_consensus.cc:348] T 00000000000000000000000000000000 P abb511ece9554e23b6d88e7ea92b9ec5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abb511ece9554e23b6d88e7ea92b9ec5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33693 } }
I20260619 02:55:31.370396 7497 raft_consensus.cc:374] T 00000000000000000000000000000000 P abb511ece9554e23b6d88e7ea92b9ec5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:31.370492 7497 raft_consensus.cc:729] T 00000000000000000000000000000000 P abb511ece9554e23b6d88e7ea92b9ec5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abb511ece9554e23b6d88e7ea92b9ec5, State: Initialized, Role: FOLLOWER
I20260619 02:55:31.371054 7497 consensus_queue.cc:260] T 00000000000000000000000000000000 P abb511ece9554e23b6d88e7ea92b9ec5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abb511ece9554e23b6d88e7ea92b9ec5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33693 } }
I20260619 02:55:31.371272 7497 raft_consensus.cc:388] T 00000000000000000000000000000000 P abb511ece9554e23b6d88e7ea92b9ec5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:55:31.371424 7497 raft_consensus.cc:482] T 00000000000000000000000000000000 P abb511ece9554e23b6d88e7ea92b9ec5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:55:31.371599 7497 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abb511ece9554e23b6d88e7ea92b9ec5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:31.373687 7497 raft_consensus.cc:504] T 00000000000000000000000000000000 P abb511ece9554e23b6d88e7ea92b9ec5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abb511ece9554e23b6d88e7ea92b9ec5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33693 } }
I20260619 02:55:31.374146 7497 leader_election.cc:302] T 00000000000000000000000000000000 P abb511ece9554e23b6d88e7ea92b9ec5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abb511ece9554e23b6d88e7ea92b9ec5; no voters:
I20260619 02:55:31.374728 7497 leader_election.cc:288] T 00000000000000000000000000000000 P abb511ece9554e23b6d88e7ea92b9ec5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:55:31.374816 7507 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abb511ece9554e23b6d88e7ea92b9ec5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:31.375422 7507 raft_consensus.cc:686] T 00000000000000000000000000000000 P abb511ece9554e23b6d88e7ea92b9ec5 [term 1 LEADER]: Becoming Leader. State: Replica: abb511ece9554e23b6d88e7ea92b9ec5, State: Running, Role: LEADER
I20260619 02:55:31.375928 7507 consensus_queue.cc:237] T 00000000000000000000000000000000 P abb511ece9554e23b6d88e7ea92b9ec5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abb511ece9554e23b6d88e7ea92b9ec5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33693 } }
I20260619 02:55:31.376662 7497 sys_catalog.cc:564] T 00000000000000000000000000000000 P abb511ece9554e23b6d88e7ea92b9ec5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:55:31.378343 7509 sys_catalog.cc:455] T 00000000000000000000000000000000 P abb511ece9554e23b6d88e7ea92b9ec5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader abb511ece9554e23b6d88e7ea92b9ec5. Latest consensus state: current_term: 1 leader_uuid: "abb511ece9554e23b6d88e7ea92b9ec5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abb511ece9554e23b6d88e7ea92b9ec5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33693 } } }
I20260619 02:55:31.378371 7508 sys_catalog.cc:455] T 00000000000000000000000000000000 P abb511ece9554e23b6d88e7ea92b9ec5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "abb511ece9554e23b6d88e7ea92b9ec5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abb511ece9554e23b6d88e7ea92b9ec5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33693 } } }
I20260619 02:55:31.378791 7508 sys_catalog.cc:458] T 00000000000000000000000000000000 P abb511ece9554e23b6d88e7ea92b9ec5 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:31.378840 7509 sys_catalog.cc:458] T 00000000000000000000000000000000 P abb511ece9554e23b6d88e7ea92b9ec5 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:31.380939 7512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:55:31.385673 7512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:55:31.393759 7512 catalog_manager.cc:1347] Generated new cluster ID: a6996cbda9d3417283f71f097a2a700d
I20260619 02:55:31.393864 7512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:55:31.429252 7512 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:55:31.429683 7512 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:55:31.436543 7512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P abb511ece9554e23b6d88e7ea92b9ec5: Generated new TSK 0
I20260619 02:55:31.437517 7512 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:55:31.457010 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 7430
W20260619 02:55:31.458173 7492 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36433
W20260619 02:55:31.458855 18968 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 7430: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 02:55:31.458981 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 7430
2026-06-19T02:55:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1773 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:55:31 dist-test-slave-9mr0 krb5kdc[7528](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:55:31 dist-test-slave-9mr0 krb5kdc[7528](info): set up 2 sockets
Jun 19 02:55:31 dist-test-slave-9mr0 krb5kdc[7528](info): commencing operation
krb5kdc: starting...
W20260619 02:55:32.466813 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.976s user 0.005s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:55:32 dist-test-slave-9mr0 krb5kdc[7528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:55:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:55:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:55:32.702090 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34129
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:33.018391 7541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:33.018692 7541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:33.018827 7541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:33.026441 7541 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:33.026572 7541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:33.026669 7541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:33.026757 7541 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:33.026823 7541 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:33.036849 7541 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34129
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:33.038130 7541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:33.039448 7541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:33.050590 7546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:33.050607 7547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:33.050800 7549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:33.051481 7541 server_base.cc:1034] running on GCE node
I20260619 02:55:33.052265 7541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:33.053735 7541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:33.054936 7541 hybrid_clock.cc:648] HybridClock initialized: now 1781837733054868 us; error 60 us; skew 500 ppm
I20260619 02:55:33.055303 7541 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:33 dist-test-slave-9mr0 krb5kdc[7528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837733, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:33.059226 7541 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:55:33.060956 7541 webserver.cc:460] Webserver started at http://127.0.0.1:34031/ using document root <none> and password file <none>
I20260619 02:55:33.061829 7541 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:33.061971 7541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:33.062342 7541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:33.065724 7541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "7e29a10e4dc54fa2b03bff09c1b14a35"
format_stamp: "Formatted at 2026-06-19 02:55:33 on dist-test-slave-9mr0"
server_key: "dc99301d0f03b13c80930b2a44be2f84"
server_key_iv: "8f97585c9fe0ee5901a7cc4533cef268"
server_key_version: "encryptionkey@0"
I20260619 02:55:33.066483 7541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "7e29a10e4dc54fa2b03bff09c1b14a35"
format_stamp: "Formatted at 2026-06-19 02:55:33 on dist-test-slave-9mr0"
server_key: "dc99301d0f03b13c80930b2a44be2f84"
server_key_iv: "8f97585c9fe0ee5901a7cc4533cef268"
server_key_version: "encryptionkey@0"
I20260619 02:55:33.071658 7541 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 02:55:33.074950 7556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:33.076223 7541 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 02:55:33.076362 7541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "7e29a10e4dc54fa2b03bff09c1b14a35"
format_stamp: "Formatted at 2026-06-19 02:55:33 on dist-test-slave-9mr0"
server_key: "dc99301d0f03b13c80930b2a44be2f84"
server_key_iv: "8f97585c9fe0ee5901a7cc4533cef268"
server_key_version: "encryptionkey@0"
I20260619 02:55:33.076548 7541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:33.094110 7541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:33.097944 7541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:33.098173 7541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:33.122669 7541 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34129
I20260619 02:55:33.122675 7607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34129 every 8 connection(s)
I20260619 02:55:33.124372 7541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:55:33.128183 7608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:33.134528 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 7541
I20260619 02:55:33.134755 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:55:33.135301 18968 external_mini_cluster.cc:1428] Setting key f6b31a3725299b16aab921006e9405ae
I20260619 02:55:33.139298 7608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e29a10e4dc54fa2b03bff09c1b14a35: Bootstrap starting.
I20260619 02:55:33.143137 7608 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e29a10e4dc54fa2b03bff09c1b14a35: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:33.144277 7608 log.cc:824] T 00000000000000000000000000000000 P 7e29a10e4dc54fa2b03bff09c1b14a35: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:33.147470 7608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e29a10e4dc54fa2b03bff09c1b14a35: No bootstrap required, opened a new log
Jun 19 02:55:33 dist-test-slave-9mr0 krb5kdc[7528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 02:55:33.156461 7608 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e29a10e4dc54fa2b03bff09c1b14a35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e29a10e4dc54fa2b03bff09c1b14a35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34129 } }
I20260619 02:55:33.156942 7608 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e29a10e4dc54fa2b03bff09c1b14a35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:33.157063 7608 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e29a10e4dc54fa2b03bff09c1b14a35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e29a10e4dc54fa2b03bff09c1b14a35, State: Initialized, Role: FOLLOWER
I20260619 02:55:33.157785 7608 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e29a10e4dc54fa2b03bff09c1b14a35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e29a10e4dc54fa2b03bff09c1b14a35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34129 } }
I20260619 02:55:33.158025 7608 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e29a10e4dc54fa2b03bff09c1b14a35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:55:33.158180 7608 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e29a10e4dc54fa2b03bff09c1b14a35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:55:33.158337 7608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e29a10e4dc54fa2b03bff09c1b14a35 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:33.158917 7611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:33.137172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:39437 (local address 127.0.0.1:34129)
0619 02:55:33.137948 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:33.137960 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:33.138001 (+ 41us) server_negotiation.cc:407] Connection header received
0619 02:55:33.139780 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:33.139808 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:33.140242 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:33.140581 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:33.142416 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:33.143820 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:33.145490 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:33.146166 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:33.146292 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:33.151140 (+ 4848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:33.151147 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:33.151165 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:33.151209 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:33.154204 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:33.155182 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:33.155189 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:33.155196 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:33.155387 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:33.155891 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:33.155921 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:33.155923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:33.156459 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:33.156707 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:33.157275 (+ 568us) server_negotiation.cc:299] Negotiation successful
0619 02:55:33.157676 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":340,"threads_started":1}
I20260619 02:55:33.160811 7608 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e29a10e4dc54fa2b03bff09c1b14a35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e29a10e4dc54fa2b03bff09c1b14a35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34129 } }
I20260619 02:55:33.161381 7608 leader_election.cc:302] T 00000000000000000000000000000000 P 7e29a10e4dc54fa2b03bff09c1b14a35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e29a10e4dc54fa2b03bff09c1b14a35; no voters:
I20260619 02:55:33.162029 7608 leader_election.cc:288] T 00000000000000000000000000000000 P 7e29a10e4dc54fa2b03bff09c1b14a35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:55:33.162142 7613 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e29a10e4dc54fa2b03bff09c1b14a35 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:33.162884 18968 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:55:33.163969 7608 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e29a10e4dc54fa2b03bff09c1b14a35 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:55:33.176442 7613 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e29a10e4dc54fa2b03bff09c1b14a35 [term 1 LEADER]: Becoming Leader. State: Replica: 7e29a10e4dc54fa2b03bff09c1b14a35, State: Running, Role: LEADER
I20260619 02:55:33.177215 7613 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e29a10e4dc54fa2b03bff09c1b14a35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e29a10e4dc54fa2b03bff09c1b14a35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34129 } }
I20260619 02:55:33.180145 7615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e29a10e4dc54fa2b03bff09c1b14a35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e29a10e4dc54fa2b03bff09c1b14a35. Latest consensus state: current_term: 1 leader_uuid: "7e29a10e4dc54fa2b03bff09c1b14a35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e29a10e4dc54fa2b03bff09c1b14a35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34129 } } }
I20260619 02:55:33.180583 7615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e29a10e4dc54fa2b03bff09c1b14a35 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:33.181108 7614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e29a10e4dc54fa2b03bff09c1b14a35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e29a10e4dc54fa2b03bff09c1b14a35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e29a10e4dc54fa2b03bff09c1b14a35" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34129 } } }
I20260619 02:55:33.181388 7614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e29a10e4dc54fa2b03bff09c1b14a35 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:33.183444 7632 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:55:33.183820 7633 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 02:55:33.187865 7632 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:55:33.194869 7611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:33.178976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:50397 (local address 127.0.0.1:34129)
0619 02:55:33.179153 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:33.179159 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:33.179615 (+ 456us) server_negotiation.cc:407] Connection header received
0619 02:55:33.179888 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:33.179905 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:33.179990 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:33.180080 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:33.182055 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:33.183081 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:33.184751 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:33.185442 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:33.185586 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:33.188467 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:33.188477 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:33.188482 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:33.188519 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:33.192040 (+ 3521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:33.192721 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:33.192726 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:33.192728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:33.192870 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:33.193273 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:33.193276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:33.193278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:33.193563 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:33.193831 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:33.194289 (+ 458us) server_negotiation.cc:299] Negotiation successful
0619 02:55:33.194483 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 02:55:33.197171 7632 catalog_manager.cc:1347] Generated new cluster ID: 3c05c307a75d4022aee352e18c92b5d7
I20260619 02:55:33.197275 7632 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:55:33.227825 7632 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:55:33.229712 7632 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:55:33.246042 7632 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e29a10e4dc54fa2b03bff09c1b14a35: Generated new TSK 0
I20260619 02:55:33.246901 7632 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:55:33.261521 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 7541
I20260619 02:55:33.262713 7603 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:33.355903 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 7541
W20260619 02:55:33.369757 7526 connection.cc:535] client connection to 127.0.0.1:34129 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34129): Connection reset by peer (error 104)
2026-06-19T02:55:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1897 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:55:33 dist-test-slave-9mr0 krb5kdc[7640](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:55:33 dist-test-slave-9mr0 krb5kdc[7640](info): set up 2 sockets
Jun 19 02:55:33 dist-test-slave-9mr0 krb5kdc[7640](info): commencing operation
krb5kdc: starting...
W20260619 02:55:34.370827 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.983s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:55:34 dist-test-slave-9mr0 krb5kdc[7640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:55:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:55:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:55:34.603566 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40391
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:34.922271 7653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:34.922577 7653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:34.922729 7653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:34.930480 7653 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:34.930610 7653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:34.930708 7653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:34.930802 7653 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:34.930860 7653 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:34.940810 7653 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:34.942081 7653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:34.943405 7653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:34.954566 7658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:34.954533 7661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:34.954499 7659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:34.955559 7653 server_base.cc:1034] running on GCE node
I20260619 02:55:34.956290 7653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:34.957672 7653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:34.958884 7653 hybrid_clock.cc:648] HybridClock initialized: now 1781837734958796 us; error 68 us; skew 500 ppm
I20260619 02:55:34.959232 7653 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:34 dist-test-slave-9mr0 krb5kdc[7640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837734, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:34.962230 7653 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:55:34.963657 7653 webserver.cc:460] Webserver started at http://127.0.0.1:36763/ using document root <none> and password file <none>
I20260619 02:55:34.964299 7653 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:34.964440 7653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:34.964721 7653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:34.967767 7653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "6b9381730af9412f81272895d69971b3"
format_stamp: "Formatted at 2026-06-19 02:55:34 on dist-test-slave-9mr0"
server_key: "b8529ad97cebf057aeebf0f837947539"
server_key_iv: "8a90182b2af49fc03284ae86b5402686"
server_key_version: "encryptionkey@0"
I20260619 02:55:34.968528 7653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "6b9381730af9412f81272895d69971b3"
format_stamp: "Formatted at 2026-06-19 02:55:34 on dist-test-slave-9mr0"
server_key: "b8529ad97cebf057aeebf0f837947539"
server_key_iv: "8a90182b2af49fc03284ae86b5402686"
server_key_version: "encryptionkey@0"
I20260619 02:55:34.973793 7653 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260619 02:55:34.977363 7668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:34.978812 7653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:55:34.979002 7653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "6b9381730af9412f81272895d69971b3"
format_stamp: "Formatted at 2026-06-19 02:55:34 on dist-test-slave-9mr0"
server_key: "b8529ad97cebf057aeebf0f837947539"
server_key_iv: "8a90182b2af49fc03284ae86b5402686"
server_key_version: "encryptionkey@0"
I20260619 02:55:34.979194 7653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:34.997089 7653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:35.001641 7653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:35.001952 7653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:35.020850 7653 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40391
I20260619 02:55:35.020922 7719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40391 every 8 connection(s)
I20260619 02:55:35.022274 7653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:55:35.025795 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 7653
I20260619 02:55:35.025970 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:55:35.026255 7720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:35.026500 18968 external_mini_cluster.cc:1428] Setting key 9278b0f356c1da7d84c1dad21dbe5f13
I20260619 02:55:35.039757 7723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:35.028254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:45247 (local address 127.0.0.1:40391)
0619 02:55:35.029141 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:35.029153 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:35.029196 (+ 43us) server_negotiation.cc:407] Connection header received
0619 02:55:35.030973 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:35.030994 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:35.031388 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:35.031726 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:35.032306 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:35.033583 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:35.035383 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:35.036056 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:35.036204 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:35.037494 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:35.037501 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:35.037514 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:35.038084 (+ 570us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:35.038158 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:35.038191 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:35.038305 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:35.038546 (+ 241us) server_negotiation.cc:299] Negotiation successful
0619 02:55:35.038758 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":382,"threads_started":1}
I20260619 02:55:35.039853 7720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b9381730af9412f81272895d69971b3: Bootstrap starting.
I20260619 02:55:35.042943 7720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b9381730af9412f81272895d69971b3: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:35.043426 18968 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:55:35.044095 7720 log.cc:824] T 00000000000000000000000000000000 P 6b9381730af9412f81272895d69971b3: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:35.047235 7720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b9381730af9412f81272895d69971b3: No bootstrap required, opened a new log
I20260619 02:55:35.054417 7723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:35.048071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:45707 (local address 127.0.0.1:40391)
0619 02:55:35.048249 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:35.048255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:35.048579 (+ 324us) server_negotiation.cc:407] Connection header received
0619 02:55:35.048753 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:35.048756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:35.048815 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:35.048895 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:35.049482 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:35.050246 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:35.052064 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:35.052533 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:35.052619 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:35.053721 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:35.053725 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:35.053729 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:35.053745 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:35.053779 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:35.053797 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:35.053877 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:35.054063 (+ 186us) server_negotiation.cc:299] Negotiation successful
0619 02:55:35.054103 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:55:35.056891 7720 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b9381730af9412f81272895d69971b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9381730af9412f81272895d69971b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40391 } }
I20260619 02:55:35.057346 7720 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b9381730af9412f81272895d69971b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:35.057483 7720 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b9381730af9412f81272895d69971b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b9381730af9412f81272895d69971b3, State: Initialized, Role: FOLLOWER
I20260619 02:55:35.058166 7720 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b9381730af9412f81272895d69971b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9381730af9412f81272895d69971b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40391 } }
I20260619 02:55:35.058449 7720 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b9381730af9412f81272895d69971b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:55:35.058588 7720 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b9381730af9412f81272895d69971b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:55:35.058785 7720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b9381730af9412f81272895d69971b3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:35.061769 7720 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b9381730af9412f81272895d69971b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9381730af9412f81272895d69971b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40391 } }
I20260619 02:55:35.062320 7720 leader_election.cc:302] T 00000000000000000000000000000000 P 6b9381730af9412f81272895d69971b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b9381730af9412f81272895d69971b3; no voters:
I20260619 02:55:35.062973 7720 leader_election.cc:288] T 00000000000000000000000000000000 P 6b9381730af9412f81272895d69971b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:55:35.063129 7730 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b9381730af9412f81272895d69971b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:35.063985 7730 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b9381730af9412f81272895d69971b3 [term 1 LEADER]: Becoming Leader. State: Replica: 6b9381730af9412f81272895d69971b3, State: Running, Role: LEADER
I20260619 02:55:35.064558 7730 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b9381730af9412f81272895d69971b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9381730af9412f81272895d69971b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40391 } }
I20260619 02:55:35.066408 7730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b9381730af9412f81272895d69971b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b9381730af9412f81272895d69971b3. Latest consensus state: current_term: 1 leader_uuid: "6b9381730af9412f81272895d69971b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9381730af9412f81272895d69971b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40391 } } }
I20260619 02:55:35.066691 7730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b9381730af9412f81272895d69971b3 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:35.067191 7732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b9381730af9412f81272895d69971b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b9381730af9412f81272895d69971b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b9381730af9412f81272895d69971b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40391 } } }
I20260619 02:55:35.067535 7732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b9381730af9412f81272895d69971b3 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:35.071027 7720 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b9381730af9412f81272895d69971b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:55:35.071188 7734 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:55:35.077546 7734 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:55:35.086707 7734 catalog_manager.cc:1347] Generated new cluster ID: 070b7de5ae034bcab00b1719cea826ed
I20260619 02:55:35.086836 7734 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:55:35.111857 7734 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:55:35.113726 7734 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:55:35.126319 7734 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b9381730af9412f81272895d69971b3: Generated new TSK 0
I20260619 02:55:35.127214 7734 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:55:35.142961 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 7653
W20260619 02:55:35.144174 7715 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.241:45247
W20260619 02:55:35.144928 18968 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 7653: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 02:55:35.145074 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 7653
W20260619 02:55:35.158170 7638 connection.cc:535] client connection to 127.0.0.1:40391 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40391): Connection reset by peer (error 104)
2026-06-19T02:55:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1788 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:55:35 dist-test-slave-9mr0 krb5kdc[7751](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:55:35 dist-test-slave-9mr0 krb5kdc[7751](info): set up 2 sockets
Jun 19 02:55:35 dist-test-slave-9mr0 krb5kdc[7751](info): commencing operation
krb5kdc: starting...
W20260619 02:55:36.158977 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.982s user 0.005s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:55:36 dist-test-slave-9mr0 krb5kdc[7751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:55:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:55:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:55:36.393862 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41071
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:36.711004 7764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:36.711264 7764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:36.711361 7764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:36.718485 7764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:36.718591 7764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:36.718662 7764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:36.718744 7764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:36.718792 7764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:36.728507 7764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:36.729746 7764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:36.731079 7764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:36.742239 7769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:36.742317 7770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:36.742280 7772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:36.743194 7764 server_base.cc:1034] running on GCE node
I20260619 02:55:36.743954 7764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:36.745309 7764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:36.746515 7764 hybrid_clock.cc:648] HybridClock initialized: now 1781837736746474 us; error 36 us; skew 500 ppm
I20260619 02:55:36.746868 7764 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:36 dist-test-slave-9mr0 krb5kdc[7751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837736, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:36.754324 7764 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:55:36.755872 7764 webserver.cc:460] Webserver started at http://127.0.0.1:42153/ using document root <none> and password file <none>
I20260619 02:55:36.756547 7764 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:36.756664 7764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:36.756950 7764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:36.760166 7764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "f9c8fd03923f4d07b5c7358611da2a95"
format_stamp: "Formatted at 2026-06-19 02:55:36 on dist-test-slave-9mr0"
server_key: "d1ec3765fefeb54b7ec8f80476050254"
server_key_iv: "e48654dfdcd1e7813cb6fbae4e17b4aa"
server_key_version: "encryptionkey@0"
I20260619 02:55:36.760964 7764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "f9c8fd03923f4d07b5c7358611da2a95"
format_stamp: "Formatted at 2026-06-19 02:55:36 on dist-test-slave-9mr0"
server_key: "d1ec3765fefeb54b7ec8f80476050254"
server_key_iv: "e48654dfdcd1e7813cb6fbae4e17b4aa"
server_key_version: "encryptionkey@0"
I20260619 02:55:36.766057 7764 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 02:55:36.769572 7779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:36.770918 7764 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:55:36.771078 7764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "f9c8fd03923f4d07b5c7358611da2a95"
format_stamp: "Formatted at 2026-06-19 02:55:36 on dist-test-slave-9mr0"
server_key: "d1ec3765fefeb54b7ec8f80476050254"
server_key_iv: "e48654dfdcd1e7813cb6fbae4e17b4aa"
server_key_version: "encryptionkey@0"
I20260619 02:55:36.771255 7764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:36.776839 7764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:36.780623 7764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:36.780879 7764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:36.798242 7764 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41071
I20260619 02:55:36.798261 7830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41071 every 8 connection(s)
I20260619 02:55:36.799651 7764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:55:36.803224 7831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:36.805805 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 7764
I20260619 02:55:36.805977 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:55:36.806489 18968 external_mini_cluster.cc:1428] Setting key fbc61d4fd4d49f6154e2d22e5c2f287e
I20260619 02:55:36.814112 7834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:36.808362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:57513 (local address 127.0.0.1:41071)
0619 02:55:36.809135 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:36.809147 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:36.809186 (+ 39us) server_negotiation.cc:407] Connection header received
0619 02:55:36.810844 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:36.810867 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:36.811212 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:36.811535 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:36.812633 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:36.812639 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:36.812651 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:36.812665 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:36.812725 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:36.812749 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:36.812915 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:36.813041 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 02:55:36.813239 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":316,"threads_started":1}
I20260619 02:55:36.814909 7831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9c8fd03923f4d07b5c7358611da2a95: Bootstrap starting.
I20260619 02:55:36.817508 18968 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:55:36.818012 7831 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9c8fd03923f4d07b5c7358611da2a95: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:36.819212 7831 log.cc:824] T 00000000000000000000000000000000 P f9c8fd03923f4d07b5c7358611da2a95: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:36.822263 7831 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9c8fd03923f4d07b5c7358611da2a95: No bootstrap required, opened a new log
I20260619 02:55:36.824715 7834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:36.822321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:45655 (local address 127.0.0.1:41071)
0619 02:55:36.822451 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:36.822455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:36.822943 (+ 488us) server_negotiation.cc:407] Connection header received
0619 02:55:36.823087 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:36.823091 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:36.823147 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:36.823240 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:36.824133 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:36.824138 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:36.824141 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:36.824159 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:36.824203 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:36.824218 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:36.824311 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:36.824414 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 02:55:36.824460 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 02:55:36.830008 7831 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9c8fd03923f4d07b5c7358611da2a95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9c8fd03923f4d07b5c7358611da2a95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41071 } }
I20260619 02:55:36.830351 7831 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9c8fd03923f4d07b5c7358611da2a95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:36.830447 7831 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9c8fd03923f4d07b5c7358611da2a95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9c8fd03923f4d07b5c7358611da2a95, State: Initialized, Role: FOLLOWER
I20260619 02:55:36.831018 7831 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9c8fd03923f4d07b5c7358611da2a95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9c8fd03923f4d07b5c7358611da2a95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41071 } }
I20260619 02:55:36.831235 7831 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9c8fd03923f4d07b5c7358611da2a95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:55:36.831362 7831 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9c8fd03923f4d07b5c7358611da2a95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:55:36.831502 7831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9c8fd03923f4d07b5c7358611da2a95 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:36.833585 7831 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9c8fd03923f4d07b5c7358611da2a95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9c8fd03923f4d07b5c7358611da2a95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41071 } }
I20260619 02:55:36.834039 7831 leader_election.cc:302] T 00000000000000000000000000000000 P f9c8fd03923f4d07b5c7358611da2a95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9c8fd03923f4d07b5c7358611da2a95; no voters:
I20260619 02:55:36.834543 7831 leader_election.cc:288] T 00000000000000000000000000000000 P f9c8fd03923f4d07b5c7358611da2a95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:55:36.834674 7841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9c8fd03923f4d07b5c7358611da2a95 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:36.835287 7841 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9c8fd03923f4d07b5c7358611da2a95 [term 1 LEADER]: Becoming Leader. State: Replica: f9c8fd03923f4d07b5c7358611da2a95, State: Running, Role: LEADER
I20260619 02:55:36.835762 7841 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9c8fd03923f4d07b5c7358611da2a95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9c8fd03923f4d07b5c7358611da2a95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41071 } }
I20260619 02:55:36.836427 7831 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9c8fd03923f4d07b5c7358611da2a95 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:55:36.838243 7842 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9c8fd03923f4d07b5c7358611da2a95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9c8fd03923f4d07b5c7358611da2a95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9c8fd03923f4d07b5c7358611da2a95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41071 } } }
I20260619 02:55:36.838671 7842 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9c8fd03923f4d07b5c7358611da2a95 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:36.839330 7843 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9c8fd03923f4d07b5c7358611da2a95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9c8fd03923f4d07b5c7358611da2a95. Latest consensus state: current_term: 1 leader_uuid: "f9c8fd03923f4d07b5c7358611da2a95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9c8fd03923f4d07b5c7358611da2a95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41071 } } }
I20260619 02:55:36.839630 7843 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9c8fd03923f4d07b5c7358611da2a95 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:36.843151 7846 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:55:36.847054 7846 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:55:36.847733 7858 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 02:55:36.855731 7846 catalog_manager.cc:1347] Generated new cluster ID: a995c68a524d40408596a7d74d147ea8
I20260619 02:55:36.855825 7846 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:55:36.869532 7846 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:55:36.870039 7846 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:55:36.878082 7846 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9c8fd03923f4d07b5c7358611da2a95: Generated new TSK 0
I20260619 02:55:36.878885 7846 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:55:36.886422 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 7764
W20260619 02:55:36.887606 7826 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.241:57513
W20260619 02:55:36.888279 18968 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 7764: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 02:55:36.888432 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 7764
2026-06-19T02:55:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1743 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:55:36 dist-test-slave-9mr0 krb5kdc[7862](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:55:36 dist-test-slave-9mr0 krb5kdc[7862](info): set up 2 sockets
Jun 19 02:55:36 dist-test-slave-9mr0 krb5kdc[7862](info): commencing operation
krb5kdc: starting...
W20260619 02:55:37.894742 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.974s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:55:38 dist-test-slave-9mr0 krb5kdc[7862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:55:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:55:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:55:38.134090 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37199
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37199
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:38.452859 7875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:38.453154 7875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:38.453279 7875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:38.460628 7875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:38.460737 7875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:38.460819 7875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:38.460898 7875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:38.460958 7875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:38.470747 7875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37199
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:38.472025 7875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:38.473284 7875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:38.484534 7880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:38.484562 7883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:38.484596 7881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:38.485549 7875 server_base.cc:1034] running on GCE node
I20260619 02:55:38.486353 7875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:38.487736 7875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:38.488937 7875 hybrid_clock.cc:648] HybridClock initialized: now 1781837738488883 us; error 48 us; skew 500 ppm
I20260619 02:55:38.489272 7875 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:38 dist-test-slave-9mr0 krb5kdc[7862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837738, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:38.492278 7875 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:55:38.493667 7875 webserver.cc:460] Webserver started at http://127.0.0.1:33207/ using document root <none> and password file <none>
I20260619 02:55:38.494318 7875 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:38.494464 7875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:38.494787 7875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:38.497748 7875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "5880944817cd4e27a6c1039b1d32be3f"
format_stamp: "Formatted at 2026-06-19 02:55:38 on dist-test-slave-9mr0"
server_key: "a77dd5d06770390456a556aaf781e139"
server_key_iv: "9c789c3afb4e8b4c9e3ed8fa85d9c4ab"
server_key_version: "encryptionkey@0"
I20260619 02:55:38.498536 7875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "5880944817cd4e27a6c1039b1d32be3f"
format_stamp: "Formatted at 2026-06-19 02:55:38 on dist-test-slave-9mr0"
server_key: "a77dd5d06770390456a556aaf781e139"
server_key_iv: "9c789c3afb4e8b4c9e3ed8fa85d9c4ab"
server_key_version: "encryptionkey@0"
I20260619 02:55:38.503580 7875 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 02:55:38.506886 7890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:38.508147 7875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 02:55:38.508304 7875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "5880944817cd4e27a6c1039b1d32be3f"
format_stamp: "Formatted at 2026-06-19 02:55:38 on dist-test-slave-9mr0"
server_key: "a77dd5d06770390456a556aaf781e139"
server_key_iv: "9c789c3afb4e8b4c9e3ed8fa85d9c4ab"
server_key_version: "encryptionkey@0"
I20260619 02:55:38.508493 7875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:38.529681 7875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:38.533921 7875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:38.534185 7875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:38.551456 7875 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37199
I20260619 02:55:38.551486 7941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37199 every 8 connection(s)
I20260619 02:55:38.552745 7875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:55:38.556407 7942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:38.557416 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 7875
I20260619 02:55:38.557567 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:55:38.558020 18968 external_mini_cluster.cc:1428] Setting key 8d57fffa4d5a132e7c8f7c80ddabcb13
I20260619 02:55:38.569938 7942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5880944817cd4e27a6c1039b1d32be3f: Bootstrap starting.
Jun 19 02:55:38 dist-test-slave-9mr0 krb5kdc[7862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 02:55:38.573796 7942 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5880944817cd4e27a6c1039b1d32be3f: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:38.574947 7942 log.cc:824] T 00000000000000000000000000000000 P 5880944817cd4e27a6c1039b1d32be3f: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:38.578194 7942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5880944817cd4e27a6c1039b1d32be3f: No bootstrap required, opened a new log
I20260619 02:55:38.581102 7945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:38.559758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:41277 (local address 127.0.0.1:37199)
0619 02:55:38.560495 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:38.560506 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:38.560549 (+ 43us) server_negotiation.cc:407] Connection header received
0619 02:55:38.562221 (+ 1672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:38.562244 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:38.562708 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:38.563049 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:38.564774 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:38.566037 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:38.567792 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:38.568461 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:38.568582 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:38.573391 (+ 4809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:38.573400 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:38.573414 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:38.573455 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:38.576777 (+ 3322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:38.577760 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:38.577765 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:38.577772 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:38.577936 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:38.578425 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:38.578429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:38.578431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:38.579001 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:38.579272 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:38.579740 (+ 468us) server_negotiation.cc:299] Negotiation successful
0619 02:55:38.580013 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":306,"threads_started":1}
I20260619 02:55:38.584813 18968 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:55:38.587236 7942 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5880944817cd4e27a6c1039b1d32be3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5880944817cd4e27a6c1039b1d32be3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37199 } }
I20260619 02:55:38.587705 7942 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5880944817cd4e27a6c1039b1d32be3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:38.587831 7942 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5880944817cd4e27a6c1039b1d32be3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5880944817cd4e27a6c1039b1d32be3f, State: Initialized, Role: FOLLOWER
I20260619 02:55:38.588593 7942 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5880944817cd4e27a6c1039b1d32be3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5880944817cd4e27a6c1039b1d32be3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37199 } }
I20260619 02:55:38.588917 7942 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5880944817cd4e27a6c1039b1d32be3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:55:38.589057 7942 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5880944817cd4e27a6c1039b1d32be3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:55:38.589227 7942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5880944817cd4e27a6c1039b1d32be3f [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:38.592482 7942 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5880944817cd4e27a6c1039b1d32be3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5880944817cd4e27a6c1039b1d32be3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37199 } }
I20260619 02:55:38.593098 7942 leader_election.cc:302] T 00000000000000000000000000000000 P 5880944817cd4e27a6c1039b1d32be3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5880944817cd4e27a6c1039b1d32be3f; no voters:
I20260619 02:55:38.593753 7942 leader_election.cc:288] T 00000000000000000000000000000000 P 5880944817cd4e27a6c1039b1d32be3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:55:38.593860 7952 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5880944817cd4e27a6c1039b1d32be3f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:38.595005 7952 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5880944817cd4e27a6c1039b1d32be3f [term 1 LEADER]: Becoming Leader. State: Replica: 5880944817cd4e27a6c1039b1d32be3f, State: Running, Role: LEADER
I20260619 02:55:38.595655 7952 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5880944817cd4e27a6c1039b1d32be3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5880944817cd4e27a6c1039b1d32be3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37199 } }
I20260619 02:55:38.596251 7942 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5880944817cd4e27a6c1039b1d32be3f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:55:38.598752 7953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5880944817cd4e27a6c1039b1d32be3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5880944817cd4e27a6c1039b1d32be3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5880944817cd4e27a6c1039b1d32be3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37199 } } }
I20260619 02:55:38.599177 7953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5880944817cd4e27a6c1039b1d32be3f [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:38.598968 7954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5880944817cd4e27a6c1039b1d32be3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5880944817cd4e27a6c1039b1d32be3f. Latest consensus state: current_term: 1 leader_uuid: "5880944817cd4e27a6c1039b1d32be3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5880944817cd4e27a6c1039b1d32be3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37199 } } }
I20260619 02:55:38.599284 7954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5880944817cd4e27a6c1039b1d32be3f [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:38.604667 7958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:55:38.609192 7945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:38.590048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:38675 (local address 127.0.0.1:37199)
0619 02:55:38.590249 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:38.590255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:38.590796 (+ 541us) server_negotiation.cc:407] Connection header received
0619 02:55:38.590960 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:38.590963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:38.591029 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:38.591117 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:38.592860 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:38.593684 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:38.595426 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:38.595978 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:38.596096 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:38.600927 (+ 4831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:38.600936 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:38.600943 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:38.600990 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:38.605038 (+ 4048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:38.605879 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:38.605885 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:38.605889 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:38.606013 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:38.606501 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:38.606506 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:38.606509 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:38.607063 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:38.607360 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:38.608607 (+ 1247us) server_negotiation.cc:299] Negotiation successful
0619 02:55:38.608797 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 02:55:38.610509 7958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:55:38.619483 7958 catalog_manager.cc:1347] Generated new cluster ID: 6e5e69f5687043b79577a9a903680885
I20260619 02:55:38.619601 7958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:55:38.632086 7958 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:55:38.633637 7958 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:55:38.641598 7958 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5880944817cd4e27a6c1039b1d32be3f: Generated new TSK 0
I20260619 02:55:38.642692 7958 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:55:38.653801 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 7875
I20260619 02:55:38.654997 7937 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:38.745090 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 7875
W20260619 02:55:38.758594 7860 connection.cc:535] client connection to 127.0.0.1:37199 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37199): Connection reset by peer (error 104)
2026-06-19T02:55:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1856 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:55:38 dist-test-slave-9mr0 krb5kdc[7974](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:55:38 dist-test-slave-9mr0 krb5kdc[7974](info): set up 2 sockets
Jun 19 02:55:38 dist-test-slave-9mr0 krb5kdc[7974](info): commencing operation
krb5kdc: starting...
W20260619 02:55:39.762876 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.986s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:55:39 dist-test-slave-9mr0 krb5kdc[7974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:55:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:55:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:55:39.997593 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43051
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43051
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:40.314206 7987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:40.314512 7987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:40.314671 7987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:40.321895 7987 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:40.322018 7987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:40.322106 7987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:40.322188 7987 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:40.322258 7987 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:40.332433 7987 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43051
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:40.333786 7987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:40.335110 7987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:40.346324 7993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:40.346369 7995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:40.346395 7992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:40.347134 7987 server_base.cc:1034] running on GCE node
I20260619 02:55:40.347862 7987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:40.349247 7987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:40.350445 7987 hybrid_clock.cc:648] HybridClock initialized: now 1781837740350395 us; error 46 us; skew 500 ppm
I20260619 02:55:40.350798 7987 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:40 dist-test-slave-9mr0 krb5kdc[7974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837740, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:40.353745 7987 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:55:40.355245 7987 webserver.cc:460] Webserver started at http://127.0.0.1:34757/ using document root <none> and password file <none>
I20260619 02:55:40.355901 7987 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:40.356014 7987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:40.356287 7987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:40.359225 7987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "6c3477ee13c04e15bb5deaae9765bf87"
format_stamp: "Formatted at 2026-06-19 02:55:40 on dist-test-slave-9mr0"
server_key: "4f7e73470ca7262a5528bf4ac1f02a73"
server_key_iv: "31042a14660f81dff6470899b911b0af"
server_key_version: "encryptionkey@0"
I20260619 02:55:40.359964 7987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "6c3477ee13c04e15bb5deaae9765bf87"
format_stamp: "Formatted at 2026-06-19 02:55:40 on dist-test-slave-9mr0"
server_key: "4f7e73470ca7262a5528bf4ac1f02a73"
server_key_iv: "31042a14660f81dff6470899b911b0af"
server_key_version: "encryptionkey@0"
I20260619 02:55:40.364871 7987 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:55:40.368141 8002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:40.369468 7987 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260619 02:55:40.369618 7987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "6c3477ee13c04e15bb5deaae9765bf87"
format_stamp: "Formatted at 2026-06-19 02:55:40 on dist-test-slave-9mr0"
server_key: "4f7e73470ca7262a5528bf4ac1f02a73"
server_key_iv: "31042a14660f81dff6470899b911b0af"
server_key_version: "encryptionkey@0"
I20260619 02:55:40.369805 7987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:40.382561 7987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:40.386112 7987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:40.386348 7987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:40.403134 7987 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43051
I20260619 02:55:40.403167 8053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43051 every 8 connection(s)
I20260619 02:55:40.404433 7987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:55:40.408061 8054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:40.409809 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 7987
I20260619 02:55:40.409991 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:55:40.410408 18968 external_mini_cluster.cc:1428] Setting key 6554596d268d0c007f029560ebda0059
I20260619 02:55:40.421466 8054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c3477ee13c04e15bb5deaae9765bf87: Bootstrap starting.
I20260619 02:55:40.423663 8057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:40.412169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:54187 (local address 127.0.0.1:43051)
0619 02:55:40.412949 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:40.412961 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:40.413008 (+ 47us) server_negotiation.cc:407] Connection header received
0619 02:55:40.414579 (+ 1571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:40.414604 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:40.415051 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:40.415365 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:40.415940 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:40.417175 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:40.419093 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:40.419861 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:40.420011 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:40.421329 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:40.421337 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:40.421350 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:40.421955 (+ 605us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:40.422040 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:40.422066 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:40.422191 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:40.422451 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 02:55:40.422623 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":335,"threads_started":1}
I20260619 02:55:40.424726 8054 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c3477ee13c04e15bb5deaae9765bf87: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:40.425776 8054 log.cc:824] T 00000000000000000000000000000000 P 6c3477ee13c04e15bb5deaae9765bf87: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:40.427350 18968 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:55:40.428897 8054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c3477ee13c04e15bb5deaae9765bf87: No bootstrap required, opened a new log
I20260619 02:55:40.438853 8057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:40.432405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:60263 (local address 127.0.0.1:43051)
0619 02:55:40.432608 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:40.432614 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:40.432844 (+ 230us) server_negotiation.cc:407] Connection header received
0619 02:55:40.432991 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:40.432996 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:40.433081 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:40.433197 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:40.433683 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:40.434915 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:40.436209 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:40.436911 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:40.437050 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:40.438073 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:40.438080 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:40.438085 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:40.438108 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:40.438169 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:40.438189 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:40.438300 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:40.438429 (+ 129us) server_negotiation.cc:299] Negotiation successful
0619 02:55:40.438485 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 02:55:40.438539 8054 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c3477ee13c04e15bb5deaae9765bf87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3477ee13c04e15bb5deaae9765bf87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43051 } }
I20260619 02:55:40.439090 8054 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c3477ee13c04e15bb5deaae9765bf87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:40.439193 8054 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c3477ee13c04e15bb5deaae9765bf87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c3477ee13c04e15bb5deaae9765bf87, State: Initialized, Role: FOLLOWER
I20260619 02:55:40.439862 8054 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c3477ee13c04e15bb5deaae9765bf87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3477ee13c04e15bb5deaae9765bf87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43051 } }
I20260619 02:55:40.440161 8054 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c3477ee13c04e15bb5deaae9765bf87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:55:40.440310 8054 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c3477ee13c04e15bb5deaae9765bf87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:55:40.440488 8054 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c3477ee13c04e15bb5deaae9765bf87 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:40.443130 8054 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c3477ee13c04e15bb5deaae9765bf87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3477ee13c04e15bb5deaae9765bf87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43051 } }
I20260619 02:55:40.443622 8054 leader_election.cc:302] T 00000000000000000000000000000000 P 6c3477ee13c04e15bb5deaae9765bf87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c3477ee13c04e15bb5deaae9765bf87; no voters:
I20260619 02:55:40.444161 8054 leader_election.cc:288] T 00000000000000000000000000000000 P 6c3477ee13c04e15bb5deaae9765bf87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:55:40.444289 8064 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c3477ee13c04e15bb5deaae9765bf87 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:40.444960 8064 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c3477ee13c04e15bb5deaae9765bf87 [term 1 LEADER]: Becoming Leader. State: Replica: 6c3477ee13c04e15bb5deaae9765bf87, State: Running, Role: LEADER
I20260619 02:55:40.445633 8064 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c3477ee13c04e15bb5deaae9765bf87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3477ee13c04e15bb5deaae9765bf87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43051 } }
I20260619 02:55:40.446322 8054 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c3477ee13c04e15bb5deaae9765bf87 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:55:40.448495 8065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c3477ee13c04e15bb5deaae9765bf87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c3477ee13c04e15bb5deaae9765bf87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3477ee13c04e15bb5deaae9765bf87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43051 } } }
I20260619 02:55:40.448884 8065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c3477ee13c04e15bb5deaae9765bf87 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:40.449345 8066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c3477ee13c04e15bb5deaae9765bf87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c3477ee13c04e15bb5deaae9765bf87. Latest consensus state: current_term: 1 leader_uuid: "6c3477ee13c04e15bb5deaae9765bf87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3477ee13c04e15bb5deaae9765bf87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43051 } } }
I20260619 02:55:40.449704 8066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c3477ee13c04e15bb5deaae9765bf87 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:40.457083 8069 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:55:40.462546 8069 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:55:40.471460 8069 catalog_manager.cc:1347] Generated new cluster ID: f39230072e954064acb6115c3df96732
I20260619 02:55:40.471589 8069 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:55:40.483863 8069 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:55:40.485111 8069 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:55:40.497046 8069 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c3477ee13c04e15bb5deaae9765bf87: Generated new TSK 0
I20260619 02:55:40.498093 8069 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:55:40.512054 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 7987
W20260619 02:55:40.513258 8049 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.241:54187
W20260619 02:55:40.514091 18968 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 7987: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 02:55:40.514218 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 7987
W20260619 02:55:40.527720 7972 connection.cc:535] client connection to 127.0.0.1:43051 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43051): Connection reset by peer (error 104)
2026-06-19T02:55:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1769 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:55:40 dist-test-slave-9mr0 krb5kdc[8085](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:55:40 dist-test-slave-9mr0 krb5kdc[8085](info): set up 2 sockets
Jun 19 02:55:40 dist-test-slave-9mr0 krb5kdc[8085](info): commencing operation
krb5kdc: starting...
W20260619 02:55:41.526883 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.980s user 0.004s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:55:41 dist-test-slave-9mr0 krb5kdc[8085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:55:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:55:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 02:55:41.766079 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42547
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42547
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:42.082211 8098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:42.082492 8098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:42.082585 8098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:42.089709 8098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:42.089823 8098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:42.089881 8098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:42.089941 8098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:42.089983 8098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:42.099867 8098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42547
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837410515220-18968-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:42.101066 8098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:42.102279 8098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:42.115113 8104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:42.115170 8103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:42.115139 8106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:42.116441 8098 server_base.cc:1034] running on GCE node
I20260619 02:55:42.117184 8098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:42.118567 8098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:42.119854 8098 hybrid_clock.cc:648] HybridClock initialized: now 1781837742119803 us; error 39 us; skew 500 ppm
I20260619 02:55:42.120204 8098 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:42 dist-test-slave-9mr0 krb5kdc[8085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837742, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:42.126963 8098 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 02:55:42.128548 8098 webserver.cc:460] Webserver started at http://127.0.0.1:37947/ using document root <none> and password file <none>
I20260619 02:55:42.129217 8098 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:42.129381 8098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:42.129670 8098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:42.132652 8098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "93e09ddef2a741278e5980f27a690e1b"
format_stamp: "Formatted at 2026-06-19 02:55:42 on dist-test-slave-9mr0"
server_key: "c6b100e467069258d2d642475f7f60d7"
server_key_iv: "f3068e3078a17f8c71a542787d5254b0"
server_key_version: "encryptionkey@0"
I20260619 02:55:42.133390 8098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "93e09ddef2a741278e5980f27a690e1b"
format_stamp: "Formatted at 2026-06-19 02:55:42 on dist-test-slave-9mr0"
server_key: "c6b100e467069258d2d642475f7f60d7"
server_key_iv: "f3068e3078a17f8c71a542787d5254b0"
server_key_version: "encryptionkey@0"
I20260619 02:55:42.138598 8098 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 02:55:42.142208 8113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:42.143756 8098 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:55:42.143950 8098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "93e09ddef2a741278e5980f27a690e1b"
format_stamp: "Formatted at 2026-06-19 02:55:42 on dist-test-slave-9mr0"
server_key: "c6b100e467069258d2d642475f7f60d7"
server_key_iv: "f3068e3078a17f8c71a542787d5254b0"
server_key_version: "encryptionkey@0"
I20260619 02:55:42.144137 8098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:42.149943 8098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:42.153957 8098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:42.154219 8098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:42.171437 8098 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42547
I20260619 02:55:42.171483 8164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42547 every 8 connection(s)
I20260619 02:55:42.172832 8098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:55:42.176939 8165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:42.178540 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 8098
I20260619 02:55:42.178753 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:55:42.179286 18968 external_mini_cluster.cc:1428] Setting key ec9b2ace4d2cb872f8fc686d75554afd
I20260619 02:55:42.187053 8168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:42.181202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:58529 (local address 127.0.0.1:42547)
0619 02:55:42.182016 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:42.182028 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:42.182068 (+ 40us) server_negotiation.cc:407] Connection header received
0619 02:55:42.183708 (+ 1640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:42.183731 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:42.184107 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:42.184487 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:42.185488 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:42.185495 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:42.185507 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:42.185523 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:42.185589 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:42.185609 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:42.185812 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:42.185942 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 02:55:42.186122 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":370,"threads_started":1}
I20260619 02:55:42.190196 8165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93e09ddef2a741278e5980f27a690e1b: Bootstrap starting.
I20260619 02:55:42.190471 18968 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:55:42.193186 8165 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93e09ddef2a741278e5980f27a690e1b: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:42.194226 8165 log.cc:824] T 00000000000000000000000000000000 P 93e09ddef2a741278e5980f27a690e1b: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:42.196956 8165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93e09ddef2a741278e5980f27a690e1b: No bootstrap required, opened a new log
I20260619 02:55:42.201220 8168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:42.198927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.241:38369 (local address 127.0.0.1:42547)
0619 02:55:42.199105 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:42.199111 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:42.199320 (+ 209us) server_negotiation.cc:407] Connection header received
0619 02:55:42.199473 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:42.199479 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:42.199543 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:42.199675 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:42.200551 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:42.200556 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:42.200561 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:42.200580 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:42.200625 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:42.200645 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:42.200784 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:42.200886 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 02:55:42.200947 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 02:55:42.205395 8165 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93e09ddef2a741278e5980f27a690e1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e09ddef2a741278e5980f27a690e1b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42547 } }
I20260619 02:55:42.205749 8165 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93e09ddef2a741278e5980f27a690e1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:42.205823 8165 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93e09ddef2a741278e5980f27a690e1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93e09ddef2a741278e5980f27a690e1b, State: Initialized, Role: FOLLOWER
I20260619 02:55:42.206385 8165 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93e09ddef2a741278e5980f27a690e1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e09ddef2a741278e5980f27a690e1b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42547 } }
I20260619 02:55:42.206607 8165 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93e09ddef2a741278e5980f27a690e1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:55:42.206751 8165 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93e09ddef2a741278e5980f27a690e1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:55:42.206877 8165 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93e09ddef2a741278e5980f27a690e1b [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:42.209232 8165 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93e09ddef2a741278e5980f27a690e1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e09ddef2a741278e5980f27a690e1b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42547 } }
I20260619 02:55:42.209707 8165 leader_election.cc:302] T 00000000000000000000000000000000 P 93e09ddef2a741278e5980f27a690e1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93e09ddef2a741278e5980f27a690e1b; no voters:
I20260619 02:55:42.210304 8165 leader_election.cc:288] T 00000000000000000000000000000000 P 93e09ddef2a741278e5980f27a690e1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:55:42.210438 8175 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93e09ddef2a741278e5980f27a690e1b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:42.211079 8175 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93e09ddef2a741278e5980f27a690e1b [term 1 LEADER]: Becoming Leader. State: Replica: 93e09ddef2a741278e5980f27a690e1b, State: Running, Role: LEADER
I20260619 02:55:42.211632 8175 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93e09ddef2a741278e5980f27a690e1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e09ddef2a741278e5980f27a690e1b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42547 } }
I20260619 02:55:42.212272 8165 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93e09ddef2a741278e5980f27a690e1b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:55:42.214406 8177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93e09ddef2a741278e5980f27a690e1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93e09ddef2a741278e5980f27a690e1b. Latest consensus state: current_term: 1 leader_uuid: "93e09ddef2a741278e5980f27a690e1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e09ddef2a741278e5980f27a690e1b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42547 } } }
I20260619 02:55:42.214793 8177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93e09ddef2a741278e5980f27a690e1b [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:42.215752 8176 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93e09ddef2a741278e5980f27a690e1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93e09ddef2a741278e5980f27a690e1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e09ddef2a741278e5980f27a690e1b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42547 } } }
I20260619 02:55:42.216123 8176 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93e09ddef2a741278e5980f27a690e1b [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:42.216481 8182 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:55:42.221494 8182 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:55:42.230762 8182 catalog_manager.cc:1347] Generated new cluster ID: 0b7cbffa2f0e4a6f9a213bf4aeef081c
I20260619 02:55:42.230868 8182 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:55:42.244593 8182 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:55:42.245044 8182 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:55:42.259007 8182 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93e09ddef2a741278e5980f27a690e1b: Generated new TSK 0
I20260619 02:55:42.259855 8182 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:55:42.270455 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 8098
W20260619 02:55:42.271608 8160 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.241:58529
W20260619 02:55:42.272373 18968 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 8098: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 02:55:42.272518 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 8098
2026-06-19T02:55:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1758 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:55:42 dist-test-slave-9mr0 krb5kdc[8196](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:55:42 dist-test-slave-9mr0 krb5kdc[8196](info): set up 2 sockets
Jun 19 02:55:42 dist-test-slave-9mr0 krb5kdc[8196](info): commencing operation
krb5kdc: starting...
W20260619 02:55:43.274923 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.971s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:55:43 dist-test-slave-9mr0 krb5kdc[8196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:55:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:55:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:55:43.509567 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:45439
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:38671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:45439
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:43.825544 8209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:43.825832 8209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:43.825965 8209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:43.833222 8209 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:43.833340 8209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:43.833415 8209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:43.833499 8209 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:43.833559 8209 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:43.843379 8209 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:38671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:45439
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:45439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:43.844673 8209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:43.845943 8209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:43.857239 8217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:43.857331 8214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:43.857312 8215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:43.858240 8209 server_base.cc:1034] running on GCE node
I20260619 02:55:43.859077 8209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:43.860431 8209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:43.861651 8209 hybrid_clock.cc:648] HybridClock initialized: now 1781837743861595 us; error 46 us; skew 500 ppm
I20260619 02:55:43.862010 8209 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:43 dist-test-slave-9mr0 krb5kdc[8196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837743, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:43.865087 8209 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:55:43.866499 8209 webserver.cc:460] Webserver started at http://127.18.134.62:43987/ using document root <none> and password file <none>
I20260619 02:55:43.867178 8209 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:43.867293 8209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:43.867568 8209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:43.870407 8209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "cdd4b5b368b640cea923612dc17572c4"
format_stamp: "Formatted at 2026-06-19 02:55:43 on dist-test-slave-9mr0"
server_key: "abe019871509ccf78e40a7a15d4fc25f"
server_key_iv: "515fb0476094a9f867c37f86069674e1"
server_key_version: "encryptionkey@0"
I20260619 02:55:43.871197 8209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "cdd4b5b368b640cea923612dc17572c4"
format_stamp: "Formatted at 2026-06-19 02:55:43 on dist-test-slave-9mr0"
server_key: "abe019871509ccf78e40a7a15d4fc25f"
server_key_iv: "515fb0476094a9f867c37f86069674e1"
server_key_version: "encryptionkey@0"
I20260619 02:55:43.876030 8209 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260619 02:55:43.879318 8224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:43.880568 8209 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:55:43.880721 8209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "cdd4b5b368b640cea923612dc17572c4"
format_stamp: "Formatted at 2026-06-19 02:55:43 on dist-test-slave-9mr0"
server_key: "abe019871509ccf78e40a7a15d4fc25f"
server_key_iv: "515fb0476094a9f867c37f86069674e1"
server_key_version: "encryptionkey@0"
I20260619 02:55:43.880930 8209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:43.911209 8209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:43.915340 8209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:43.915609 8209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:43.932876 8209 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:45439
I20260619 02:55:43.932902 8275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:45439 every 8 connection(s)
I20260619 02:55:43.934139 8209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:55:43.937644 8276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:43.942590 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 8209
I20260619 02:55:43.942801 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:55:43.943310 18968 external_mini_cluster.cc:1428] Setting key 81ca33ad3f23e6dda46a8d8b7765e875
I20260619 02:55:43.948398 8276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4: Bootstrap starting.
I20260619 02:55:43.951900 8276 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:43.953109 8276 log.cc:824] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:43.956498 8276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4: No bootstrap required, opened a new log
Jun 19 02:55:43 dist-test-slave-9mr0 krb5kdc[8196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:55:43.964054 8276 raft_consensus.cc:348] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdd4b5b368b640cea923612dc17572c4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 45439 } }
I20260619 02:55:43.964440 8276 raft_consensus.cc:374] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:43.964541 8276 raft_consensus.cc:729] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdd4b5b368b640cea923612dc17572c4, State: Initialized, Role: FOLLOWER
I20260619 02:55:43.965317 8276 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdd4b5b368b640cea923612dc17572c4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 45439 } }
I20260619 02:55:43.965556 8281 raft_consensus.cc:482] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 02:55:43.965804 8281 raft_consensus.cc:504] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdd4b5b368b640cea923612dc17572c4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 45439 } }
I20260619 02:55:43.966302 8279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:43.945013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44999 (local address 127.18.134.62:45439)
0619 02:55:43.945785 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:43.945808 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:43.945852 (+ 44us) server_negotiation.cc:407] Connection header received
0619 02:55:43.947549 (+ 1697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:43.947572 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:43.947959 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:43.948258 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:43.950009 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:43.951371 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:43.953173 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:43.954095 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:43.954299 (+ 204us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:43.958769 (+ 4470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:43.958779 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:43.958794 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:43.958833 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:43.961772 (+ 2939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:43.962506 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:43.962513 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:43.962519 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:43.962745 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:43.963167 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:43.963171 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:43.963173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:43.963693 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:43.963975 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:43.964458 (+ 483us) server_negotiation.cc:299] Negotiation successful
0619 02:55:43.964988 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":344,"threads_started":1}
I20260619 02:55:43.966427 8276 raft_consensus.cc:388] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:55:43.966624 8276 raft_consensus.cc:482] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:55:43.966451 8281 leader_election.cc:302] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cdd4b5b368b640cea923612dc17572c4; no voters:
I20260619 02:55:43.966796 8276 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:43.967330 8281 leader_election.cc:288] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260619 02:55:43.969517 8276 raft_consensus.cc:504] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdd4b5b368b640cea923612dc17572c4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 45439 } }
I20260619 02:55:43.969754 8276 leader_election.cc:302] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cdd4b5b368b640cea923612dc17572c4; no voters:
I20260619 02:55:43.969812 8282 raft_consensus.cc:2741] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260619 02:55:43.970012 8276 leader_election.cc:288] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:55:43.970391 8281 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:43.970556 18968 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:55:43.971894 8276 sys_catalog.cc:564] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:55:43.974342 8281 raft_consensus.cc:686] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4 [term 1 LEADER]: Becoming Leader. State: Replica: cdd4b5b368b640cea923612dc17572c4, State: Running, Role: LEADER
I20260619 02:55:43.975052 8281 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdd4b5b368b640cea923612dc17572c4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 45439 } }
I20260619 02:55:43.977288 8283 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cdd4b5b368b640cea923612dc17572c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdd4b5b368b640cea923612dc17572c4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 45439 } } }
I20260619 02:55:43.977710 8283 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:43.988116 8282 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdd4b5b368b640cea923612dc17572c4. Latest consensus state: current_term: 1 leader_uuid: "cdd4b5b368b640cea923612dc17572c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdd4b5b368b640cea923612dc17572c4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 45439 } } }
I20260619 02:55:43.988451 8282 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:43.988557 8294 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:55:43.991997 8279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:43.974704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54569 (local address 127.18.134.62:45439)
0619 02:55:43.974894 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:43.974901 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:43.975350 (+ 449us) server_negotiation.cc:407] Connection header received
0619 02:55:43.975521 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:43.975526 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:43.975610 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:43.975720 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:43.977337 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:43.978457 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:43.980344 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:43.981024 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:43.981116 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:43.984694 (+ 3578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:43.984701 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:43.984705 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:43.984741 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:43.988721 (+ 3980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:43.989644 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:43.989649 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:43.989652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:43.989796 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:43.990290 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:43.990295 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:43.990298 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:43.990611 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:43.990905 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:43.991419 (+ 514us) server_negotiation.cc:299] Negotiation successful
0619 02:55:43.991617 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 02:55:43.995715 8294 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:55:44.004462 8294 catalog_manager.cc:1347] Generated new cluster ID: 23f86925c4364b569686c69d0443aaa4
I20260619 02:55:44.004591 8294 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:55:44.043550 8294 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:55:44.044818 8294 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:55:44.050163 8294 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cdd4b5b368b640cea923612dc17572c4: Generated new TSK 0
I20260619 02:55:44.050978 8294 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:55:44.057199 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 8209
I20260619 02:55:44.058427 8271 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:44.148993 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 8209
W20260619 02:55:44.162936 8194 connection.cc:535] client connection to 127.18.134.62:45439 recv error: Network error: failed to read from TLS socket (remote: 127.18.134.62:45439): Connection reset by peer (error 104)
2026-06-19T02:55:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1877 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:55:44 dist-test-slave-9mr0 krb5kdc[8309](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:55:44 dist-test-slave-9mr0 krb5kdc[8309](info): set up 2 sockets
Jun 19 02:55:44 dist-test-slave-9mr0 krb5kdc[8309](info): commencing operation
krb5kdc: starting...
W20260619 02:55:45.162950 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.981s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:55:45 dist-test-slave-9mr0 krb5kdc[8309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:55:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:55:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:55:45.411911 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:35019
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:45137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:35019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:45.724313 8322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:45.724596 8322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:45.724695 8322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:45.732191 8322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:45.732336 8322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:45.732417 8322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:45.732507 8322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:45.732569 8322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:45.743219 8322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:45137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:35019
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:35019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:45.744562 8322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:45.745847 8322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:45.757359 8327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:45.757293 8328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:45.757320 8330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:45.757987 8322 server_base.cc:1034] running on GCE node
I20260619 02:55:45.758806 8322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:45.760223 8322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:45.761451 8322 hybrid_clock.cc:648] HybridClock initialized: now 1781837745761409 us; error 33 us; skew 500 ppm
I20260619 02:55:45.761817 8322 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:45 dist-test-slave-9mr0 krb5kdc[8309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837745, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:45.764914 8322 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:55:45.766352 8322 webserver.cc:460] Webserver started at http://127.18.134.62:36891/ using document root <none> and password file <none>
I20260619 02:55:45.767068 8322 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:45.767170 8322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:45.767436 8322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:45.770300 8322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "334be13fc64844ba853ac4ecee438472"
format_stamp: "Formatted at 2026-06-19 02:55:45 on dist-test-slave-9mr0"
server_key: "2a3e6670a8fb0c4a08b2faa4abfa0bd9"
server_key_iv: "99007ae6cade19ecc31e040dd8a24a89"
server_key_version: "encryptionkey@0"
I20260619 02:55:45.771032 8322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "334be13fc64844ba853ac4ecee438472"
format_stamp: "Formatted at 2026-06-19 02:55:45 on dist-test-slave-9mr0"
server_key: "2a3e6670a8fb0c4a08b2faa4abfa0bd9"
server_key_iv: "99007ae6cade19ecc31e040dd8a24a89"
server_key_version: "encryptionkey@0"
I20260619 02:55:45.776351 8322 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260619 02:55:45.779948 8337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:45.781327 8322 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:55:45.781494 8322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "334be13fc64844ba853ac4ecee438472"
format_stamp: "Formatted at 2026-06-19 02:55:45 on dist-test-slave-9mr0"
server_key: "2a3e6670a8fb0c4a08b2faa4abfa0bd9"
server_key_iv: "99007ae6cade19ecc31e040dd8a24a89"
server_key_version: "encryptionkey@0"
I20260619 02:55:45.781690 8322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:45.808634 8322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:45.813023 8322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:45.813275 8322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:45.830682 8322 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:35019
I20260619 02:55:45.830703 8388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:35019 every 8 connection(s)
I20260619 02:55:45.832065 8322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:55:45.834365 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 8322
I20260619 02:55:45.834582 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:55:45.835175 18968 external_mini_cluster.cc:1428] Setting key 00144c5a82d126602298d08e81d021f3
I20260619 02:55:45.836246 8389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:45.848304 8392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:45.837166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46039 (local address 127.18.134.62:35019)
0619 02:55:45.837922 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:45.837933 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:45.837972 (+ 39us) server_negotiation.cc:407] Connection header received
0619 02:55:45.839506 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:45.839526 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:45.839917 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:45.840233 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:45.840780 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:45.842093 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:45.843925 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:45.844582 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:45.844756 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:45.845965 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:45.845973 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:45.845987 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:45.846549 (+ 562us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:45.846654 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:45.846700 (+ 46us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:45.846839 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:45.847106 (+ 267us) server_negotiation.cc:299] Negotiation successful
0619 02:55:45.847288 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":332,"threads_started":1}
I20260619 02:55:45.849385 8389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 334be13fc64844ba853ac4ecee438472: Bootstrap starting.
I20260619 02:55:45.851989 18968 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:55:45.852586 8389 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 334be13fc64844ba853ac4ecee438472: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:45.853781 8389 log.cc:824] T 00000000000000000000000000000000 P 334be13fc64844ba853ac4ecee438472: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:45.856794 8389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 334be13fc64844ba853ac4ecee438472: No bootstrap required, opened a new log
I20260619 02:55:45.865832 8392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:45.856746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55931 (local address 127.18.134.62:35019)
0619 02:55:45.856947 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:45.856955 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:45.857494 (+ 539us) server_negotiation.cc:407] Connection header received
0619 02:55:45.857714 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:45.857718 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:45.857780 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:45.857856 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:45.858552 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:45.859379 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:45.861412 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:45.861869 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:45.861962 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:45.864964 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:45.864970 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:45.864974 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:45.864993 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:45.865042 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:45.865058 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:45.865174 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:45.865410 (+ 236us) server_negotiation.cc:299] Negotiation successful
0619 02:55:45.865459 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 02:55:45.866917 8389 raft_consensus.cc:348] T 00000000000000000000000000000000 P 334be13fc64844ba853ac4ecee438472 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "334be13fc64844ba853ac4ecee438472" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35019 } }
I20260619 02:55:45.867369 8389 raft_consensus.cc:374] T 00000000000000000000000000000000 P 334be13fc64844ba853ac4ecee438472 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:45.867476 8389 raft_consensus.cc:729] T 00000000000000000000000000000000 P 334be13fc64844ba853ac4ecee438472 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 334be13fc64844ba853ac4ecee438472, State: Initialized, Role: FOLLOWER
I20260619 02:55:45.868111 8389 consensus_queue.cc:260] T 00000000000000000000000000000000 P 334be13fc64844ba853ac4ecee438472 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "334be13fc64844ba853ac4ecee438472" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35019 } }
I20260619 02:55:45.868352 8389 raft_consensus.cc:388] T 00000000000000000000000000000000 P 334be13fc64844ba853ac4ecee438472 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:55:45.868508 8389 raft_consensus.cc:482] T 00000000000000000000000000000000 P 334be13fc64844ba853ac4ecee438472 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:55:45.868686 8389 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 334be13fc64844ba853ac4ecee438472 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:45.871317 8389 raft_consensus.cc:504] T 00000000000000000000000000000000 P 334be13fc64844ba853ac4ecee438472 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "334be13fc64844ba853ac4ecee438472" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35019 } }
I20260619 02:55:45.871831 8389 leader_election.cc:302] T 00000000000000000000000000000000 P 334be13fc64844ba853ac4ecee438472 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 334be13fc64844ba853ac4ecee438472; no voters:
I20260619 02:55:45.872426 8389 leader_election.cc:288] T 00000000000000000000000000000000 P 334be13fc64844ba853ac4ecee438472 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:55:45.872536 8399 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 334be13fc64844ba853ac4ecee438472 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:45.873167 8399 raft_consensus.cc:686] T 00000000000000000000000000000000 P 334be13fc64844ba853ac4ecee438472 [term 1 LEADER]: Becoming Leader. State: Replica: 334be13fc64844ba853ac4ecee438472, State: Running, Role: LEADER
I20260619 02:55:45.873737 8399 consensus_queue.cc:237] T 00000000000000000000000000000000 P 334be13fc64844ba853ac4ecee438472 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "334be13fc64844ba853ac4ecee438472" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35019 } }
I20260619 02:55:45.874537 8389 sys_catalog.cc:564] T 00000000000000000000000000000000 P 334be13fc64844ba853ac4ecee438472 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:55:45.876705 8400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 334be13fc64844ba853ac4ecee438472 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "334be13fc64844ba853ac4ecee438472" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "334be13fc64844ba853ac4ecee438472" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35019 } } }
I20260619 02:55:45.876678 8401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 334be13fc64844ba853ac4ecee438472 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 334be13fc64844ba853ac4ecee438472. Latest consensus state: current_term: 1 leader_uuid: "334be13fc64844ba853ac4ecee438472" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "334be13fc64844ba853ac4ecee438472" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35019 } } }
I20260619 02:55:45.882783 8400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 334be13fc64844ba853ac4ecee438472 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:45.882797 8401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 334be13fc64844ba853ac4ecee438472 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:45.891134 8407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:55:45.895987 8407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:55:45.904268 8407 catalog_manager.cc:1347] Generated new cluster ID: 886b7b3587684d92ad41d22843e9f939
I20260619 02:55:45.904376 8407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:55:45.936837 8407 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:55:45.938073 8407 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:55:45.946825 8407 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 334be13fc64844ba853ac4ecee438472: Generated new TSK 0
I20260619 02:55:45.947654 8407 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:55:45.957676 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 8322
W20260619 02:55:45.959203 8384 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46039
W20260619 02:55:45.959880 18968 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 8322: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 02:55:45.959997 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 8322
W20260619 02:55:45.973281 8307 connection.cc:535] client connection to 127.18.134.62:35019 recv error: Network error: failed to read from TLS socket (remote: 127.18.134.62:35019): Connection reset by peer (error 104)
2026-06-19T02:55:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1809 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:55:46 dist-test-slave-9mr0 krb5kdc[8420](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:55:46 dist-test-slave-9mr0 krb5kdc[8420](info): set up 2 sockets
Jun 19 02:55:46 dist-test-slave-9mr0 krb5kdc[8420](info): commencing operation
krb5kdc: starting...
W20260619 02:55:46.986867 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.995s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:55:47 dist-test-slave-9mr0 krb5kdc[8420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:55:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:55:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:55:47.229643 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:39401
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:32877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:39401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:47.545693 8433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:47.546000 8433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:47.546135 8433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:47.553488 8433 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:47.553617 8433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:47.553699 8433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:47.553777 8433 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:47.553834 8433 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:47.563803 8433 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:32877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:39401
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:39401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:47.565071 8433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:47.566354 8433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:47.577472 8438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:47.577558 8439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:47.577581 8441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:47.578388 8433 server_base.cc:1034] running on GCE node
I20260619 02:55:47.579162 8433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:47.580533 8433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:47.581736 8433 hybrid_clock.cc:648] HybridClock initialized: now 1781837747581678 us; error 53 us; skew 500 ppm
I20260619 02:55:47.582078 8433 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:47 dist-test-slave-9mr0 krb5kdc[8420](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837747, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:47.588786 8433 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:55:47.590235 8433 webserver.cc:460] Webserver started at http://127.18.134.62:35743/ using document root <none> and password file <none>
I20260619 02:55:47.590968 8433 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:47.591089 8433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:47.591413 8433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:47.594288 8433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "24b4cbc4344f4c92bd8f3d719e7dc9fc"
format_stamp: "Formatted at 2026-06-19 02:55:47 on dist-test-slave-9mr0"
server_key: "e65395ef18fddbd0a6ba55347ee517e7"
server_key_iv: "e7e9f648a622985d680942fa518c6764"
server_key_version: "encryptionkey@0"
I20260619 02:55:47.595074 8433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "24b4cbc4344f4c92bd8f3d719e7dc9fc"
format_stamp: "Formatted at 2026-06-19 02:55:47 on dist-test-slave-9mr0"
server_key: "e65395ef18fddbd0a6ba55347ee517e7"
server_key_iv: "e7e9f648a622985d680942fa518c6764"
server_key_version: "encryptionkey@0"
I20260619 02:55:47.600140 8433 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260619 02:55:47.603681 8448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:47.605010 8433 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 02:55:47.605195 8433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "24b4cbc4344f4c92bd8f3d719e7dc9fc"
format_stamp: "Formatted at 2026-06-19 02:55:47 on dist-test-slave-9mr0"
server_key: "e65395ef18fddbd0a6ba55347ee517e7"
server_key_iv: "e7e9f648a622985d680942fa518c6764"
server_key_version: "encryptionkey@0"
I20260619 02:55:47.605378 8433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:47.610693 8433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:47.614344 8433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:47.614595 8433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:47.631547 8433 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:39401
I20260619 02:55:47.631575 8499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:39401 every 8 connection(s)
I20260619 02:55:47.632931 8433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:55:47.636866 8500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:47.642325 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 8433
I20260619 02:55:47.642506 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:55:47.643019 18968 external_mini_cluster.cc:1428] Setting key cc79bfc532d7f1fa8c907f1e54cf3dcd
I20260619 02:55:47.648597 8500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24b4cbc4344f4c92bd8f3d719e7dc9fc: Bootstrap starting.
I20260619 02:55:47.650842 8503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:47.644780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55441 (local address 127.18.134.62:39401)
0619 02:55:47.645580 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:47.645591 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:47.645631 (+ 40us) server_negotiation.cc:407] Connection header received
0619 02:55:47.647663 (+ 2032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:47.647686 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:47.648051 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:47.648384 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:47.649328 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:47.649335 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:47.649347 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:47.649360 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:47.649418 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:47.649444 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:47.649604 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:47.649697 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 02:55:47.649855 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":375,"threads_started":1}
I20260619 02:55:47.652237 8500 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24b4cbc4344f4c92bd8f3d719e7dc9fc: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:47.653353 8500 log.cc:824] T 00000000000000000000000000000000 P 24b4cbc4344f4c92bd8f3d719e7dc9fc: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:47.654368 18968 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:55:47.656510 8500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24b4cbc4344f4c92bd8f3d719e7dc9fc: No bootstrap required, opened a new log
I20260619 02:55:47.662024 8503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:47.659550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52385 (local address 127.18.134.62:39401)
0619 02:55:47.659775 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:47.659781 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:47.660106 (+ 325us) server_negotiation.cc:407] Connection header received
0619 02:55:47.660318 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:47.660322 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:47.660371 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:47.660462 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:47.661373 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:47.661381 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:47.661384 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:47.661399 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:47.661441 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:47.661459 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:47.661552 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:47.661669 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 02:55:47.661715 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260619 02:55:47.665531 8500 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24b4cbc4344f4c92bd8f3d719e7dc9fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24b4cbc4344f4c92bd8f3d719e7dc9fc" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39401 } }
I20260619 02:55:47.665875 8500 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24b4cbc4344f4c92bd8f3d719e7dc9fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:47.665952 8500 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24b4cbc4344f4c92bd8f3d719e7dc9fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24b4cbc4344f4c92bd8f3d719e7dc9fc, State: Initialized, Role: FOLLOWER
I20260619 02:55:47.666448 8500 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24b4cbc4344f4c92bd8f3d719e7dc9fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24b4cbc4344f4c92bd8f3d719e7dc9fc" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39401 } }
I20260619 02:55:47.666682 8500 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24b4cbc4344f4c92bd8f3d719e7dc9fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:55:47.666795 8500 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24b4cbc4344f4c92bd8f3d719e7dc9fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:55:47.666925 8500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24b4cbc4344f4c92bd8f3d719e7dc9fc [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:47.669078 8500 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24b4cbc4344f4c92bd8f3d719e7dc9fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24b4cbc4344f4c92bd8f3d719e7dc9fc" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39401 } }
I20260619 02:55:47.669641 8500 leader_election.cc:302] T 00000000000000000000000000000000 P 24b4cbc4344f4c92bd8f3d719e7dc9fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24b4cbc4344f4c92bd8f3d719e7dc9fc; no voters:
I20260619 02:55:47.670193 8500 leader_election.cc:288] T 00000000000000000000000000000000 P 24b4cbc4344f4c92bd8f3d719e7dc9fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:55:47.670310 8510 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24b4cbc4344f4c92bd8f3d719e7dc9fc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:47.670956 8510 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24b4cbc4344f4c92bd8f3d719e7dc9fc [term 1 LEADER]: Becoming Leader. State: Replica: 24b4cbc4344f4c92bd8f3d719e7dc9fc, State: Running, Role: LEADER
I20260619 02:55:47.671489 8510 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24b4cbc4344f4c92bd8f3d719e7dc9fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24b4cbc4344f4c92bd8f3d719e7dc9fc" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39401 } }
I20260619 02:55:47.672180 8500 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24b4cbc4344f4c92bd8f3d719e7dc9fc [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:55:47.674153 8511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24b4cbc4344f4c92bd8f3d719e7dc9fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24b4cbc4344f4c92bd8f3d719e7dc9fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24b4cbc4344f4c92bd8f3d719e7dc9fc" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39401 } } }
I20260619 02:55:47.674539 8511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24b4cbc4344f4c92bd8f3d719e7dc9fc [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:47.675314 8512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24b4cbc4344f4c92bd8f3d719e7dc9fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24b4cbc4344f4c92bd8f3d719e7dc9fc. Latest consensus state: current_term: 1 leader_uuid: "24b4cbc4344f4c92bd8f3d719e7dc9fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24b4cbc4344f4c92bd8f3d719e7dc9fc" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 39401 } } }
I20260619 02:55:47.675594 8512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24b4cbc4344f4c92bd8f3d719e7dc9fc [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:47.679818 8515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:55:47.684980 8515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:55:47.694099 8515 catalog_manager.cc:1347] Generated new cluster ID: 897dd2caaa48418680e00ba2811d6052
I20260619 02:55:47.694249 8515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:55:47.704010 8515 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:55:47.704386 8515 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:55:47.717317 8515 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24b4cbc4344f4c92bd8f3d719e7dc9fc: Generated new TSK 0
I20260619 02:55:47.718130 8515 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:55:47.731508 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 8433
W20260619 02:55:47.732824 8495 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55441
W20260619 02:55:47.733538 18968 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 8433: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 02:55:47.733678 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 8433
2026-06-19T02:55:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1774 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:55:47 dist-test-slave-9mr0 krb5kdc[8531](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:55:47 dist-test-slave-9mr0 krb5kdc[8531](info): set up 2 sockets
Jun 19 02:55:47 dist-test-slave-9mr0 krb5kdc[8531](info): commencing operation
krb5kdc: starting...
W20260619 02:55:48.747192 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.981s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:55:48 dist-test-slave-9mr0 krb5kdc[8531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:55:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:55:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:55:48.990109 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:45881
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:34897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:45881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:49.311779 8544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:49.312091 8544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:49.312232 8544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:49.319545 8544 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:49.319677 8544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:49.319764 8544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:49.319850 8544 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:49.319906 8544 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:49.329679 8544 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:34897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:45881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:45881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:49.331033 8544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:49.332350 8544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:49.343596 8549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:49.343653 8550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:49.343837 8552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:49.344400 8544 server_base.cc:1034] running on GCE node
I20260619 02:55:49.345111 8544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:49.346474 8544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:49.347666 8544 hybrid_clock.cc:648] HybridClock initialized: now 1781837749347625 us; error 34 us; skew 500 ppm
I20260619 02:55:49.348013 8544 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:49 dist-test-slave-9mr0 krb5kdc[8531](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837749, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:49.350958 8544 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:55:49.352463 8544 webserver.cc:460] Webserver started at http://127.18.134.62:43793/ using document root <none> and password file <none>
I20260619 02:55:49.353120 8544 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:49.353238 8544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:49.353511 8544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:49.356395 8544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "a515beeaad9c48728a1b9e6f568da417"
format_stamp: "Formatted at 2026-06-19 02:55:49 on dist-test-slave-9mr0"
server_key: "a35dca8e9382b5a17cab16d789a111ac"
server_key_iv: "9be0dbe12f1625ba1d584418a17af458"
server_key_version: "encryptionkey@0"
I20260619 02:55:49.357141 8544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "a515beeaad9c48728a1b9e6f568da417"
format_stamp: "Formatted at 2026-06-19 02:55:49 on dist-test-slave-9mr0"
server_key: "a35dca8e9382b5a17cab16d789a111ac"
server_key_iv: "9be0dbe12f1625ba1d584418a17af458"
server_key_version: "encryptionkey@0"
I20260619 02:55:49.362116 8544 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260619 02:55:49.365437 8559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:49.366741 8544 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 02:55:49.366899 8544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "a515beeaad9c48728a1b9e6f568da417"
format_stamp: "Formatted at 2026-06-19 02:55:49 on dist-test-slave-9mr0"
server_key: "a35dca8e9382b5a17cab16d789a111ac"
server_key_iv: "9be0dbe12f1625ba1d584418a17af458"
server_key_version: "encryptionkey@0"
I20260619 02:55:49.367069 8544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:49.384254 8544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:49.388406 8544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:49.388684 8544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:49.406131 8544 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:45881
I20260619 02:55:49.406162 8610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:45881 every 8 connection(s)
I20260619 02:55:49.407439 8544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:55:49.411139 8611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:49.413131 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 8544
I20260619 02:55:49.413293 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:55:49.413686 18968 external_mini_cluster.cc:1428] Setting key 8977e0a4b9a89f8b56813cfda38b3b86
I20260619 02:55:49.422565 8611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a515beeaad9c48728a1b9e6f568da417: Bootstrap starting.
I20260619 02:55:49.426192 8611 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a515beeaad9c48728a1b9e6f568da417: Neither blocks nor log segments found. Creating new log.
Jun 19 02:55:49 dist-test-slave-9mr0 krb5kdc[8531](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:55:49.427322 8611 log.cc:824] T 00000000000000000000000000000000 P a515beeaad9c48728a1b9e6f568da417: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:49.429997 8611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a515beeaad9c48728a1b9e6f568da417: No bootstrap required, opened a new log
I20260619 02:55:49.435284 8614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:49.415320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54523 (local address 127.18.134.62:45881)
0619 02:55:49.416057 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:49.416068 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:49.416108 (+ 40us) server_negotiation.cc:407] Connection header received
0619 02:55:49.417753 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:49.417773 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:49.418141 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:49.418453 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:49.420150 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:49.421467 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:49.423138 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:49.423781 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:49.423912 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:49.428371 (+ 4459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:49.428379 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:49.428394 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:49.428433 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:49.431397 (+ 2964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:49.432038 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:49.432044 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:49.432049 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:49.432229 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:49.432597 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:49.432601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:49.432603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:49.433123 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:49.433437 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:49.433843 (+ 406us) server_negotiation.cc:299] Negotiation successful
0619 02:55:49.434252 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":329,"threads_started":1}
I20260619 02:55:49.437775 8611 raft_consensus.cc:348] T 00000000000000000000000000000000 P a515beeaad9c48728a1b9e6f568da417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a515beeaad9c48728a1b9e6f568da417" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 45881 } }
I20260619 02:55:49.438169 8611 raft_consensus.cc:374] T 00000000000000000000000000000000 P a515beeaad9c48728a1b9e6f568da417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:49.438270 8611 raft_consensus.cc:729] T 00000000000000000000000000000000 P a515beeaad9c48728a1b9e6f568da417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a515beeaad9c48728a1b9e6f568da417, State: Initialized, Role: FOLLOWER
I20260619 02:55:49.438858 18968 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:55:49.438838 8611 consensus_queue.cc:260] T 00000000000000000000000000000000 P a515beeaad9c48728a1b9e6f568da417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a515beeaad9c48728a1b9e6f568da417" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 45881 } }
I20260619 02:55:49.439082 8611 raft_consensus.cc:388] T 00000000000000000000000000000000 P a515beeaad9c48728a1b9e6f568da417 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:55:49.439200 8611 raft_consensus.cc:482] T 00000000000000000000000000000000 P a515beeaad9c48728a1b9e6f568da417 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:55:49.439349 8611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a515beeaad9c48728a1b9e6f568da417 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:49.441443 8611 raft_consensus.cc:504] T 00000000000000000000000000000000 P a515beeaad9c48728a1b9e6f568da417 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a515beeaad9c48728a1b9e6f568da417" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 45881 } }
I20260619 02:55:49.441959 8611 leader_election.cc:302] T 00000000000000000000000000000000 P a515beeaad9c48728a1b9e6f568da417 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a515beeaad9c48728a1b9e6f568da417; no voters:
I20260619 02:55:49.442541 8611 leader_election.cc:288] T 00000000000000000000000000000000 P a515beeaad9c48728a1b9e6f568da417 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:55:49.442674 8620 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a515beeaad9c48728a1b9e6f568da417 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:49.448093 8620 raft_consensus.cc:686] T 00000000000000000000000000000000 P a515beeaad9c48728a1b9e6f568da417 [term 1 LEADER]: Becoming Leader. State: Replica: a515beeaad9c48728a1b9e6f568da417, State: Running, Role: LEADER
I20260619 02:55:49.448519 8611 sys_catalog.cc:564] T 00000000000000000000000000000000 P a515beeaad9c48728a1b9e6f568da417 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:55:49.448712 8620 consensus_queue.cc:237] T 00000000000000000000000000000000 P a515beeaad9c48728a1b9e6f568da417 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a515beeaad9c48728a1b9e6f568da417" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 45881 } }
I20260619 02:55:49.451442 8621 sys_catalog.cc:455] T 00000000000000000000000000000000 P a515beeaad9c48728a1b9e6f568da417 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a515beeaad9c48728a1b9e6f568da417" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a515beeaad9c48728a1b9e6f568da417" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 45881 } } }
I20260619 02:55:49.451810 8621 sys_catalog.cc:458] T 00000000000000000000000000000000 P a515beeaad9c48728a1b9e6f568da417 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:49.453930 8623 sys_catalog.cc:455] T 00000000000000000000000000000000 P a515beeaad9c48728a1b9e6f568da417 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a515beeaad9c48728a1b9e6f568da417. Latest consensus state: current_term: 1 leader_uuid: "a515beeaad9c48728a1b9e6f568da417" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a515beeaad9c48728a1b9e6f568da417" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 45881 } } }
I20260619 02:55:49.454231 8623 sys_catalog.cc:458] T 00000000000000000000000000000000 P a515beeaad9c48728a1b9e6f568da417 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:49.456929 8626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:55:49.464555 8614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:49.443101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52489 (local address 127.18.134.62:45881)
0619 02:55:49.443279 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:49.443285 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:49.447724 (+ 4439us) server_negotiation.cc:407] Connection header received
0619 02:55:49.447884 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:49.447889 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:49.447975 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:49.448071 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:49.449652 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:49.450501 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:49.452318 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:49.453054 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:49.453181 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:49.456602 (+ 3421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:49.456609 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:49.456613 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:49.456651 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:49.460495 (+ 3844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:49.461281 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:49.461287 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:49.461291 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:49.461454 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:49.461938 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:49.461943 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:49.461946 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:49.462387 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:49.463647 (+ 1260us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:49.463802 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 02:55:49.464044 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 02:55:49.465479 8626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:55:49.476213 8626 catalog_manager.cc:1347] Generated new cluster ID: 5ec62ae832cd499d88f2cad670018162
I20260619 02:55:49.476346 8626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:55:49.485008 8626 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:55:49.486868 8626 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:55:49.505031 8626 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a515beeaad9c48728a1b9e6f568da417: Generated new TSK 0
I20260619 02:55:49.505870 8626 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:55:49.519236 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 8544
I20260619 02:55:49.520292 8606 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:49.610771 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 8544
2026-06-19T02:55:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1876 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:55:49 dist-test-slave-9mr0 krb5kdc[8644](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:55:49 dist-test-slave-9mr0 krb5kdc[8644](info): set up 2 sockets
Jun 19 02:55:49 dist-test-slave-9mr0 krb5kdc[8644](info): commencing operation
krb5kdc: starting...
W20260619 02:55:50.623055 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.981s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:55:50 dist-test-slave-9mr0 krb5kdc[8644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:55:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:55:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:55:50.865998 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:36509
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:40889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:36509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:51.186177 8657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:51.186486 8657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:51.186580 8657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:51.193948 8657 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:51.194072 8657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:51.194119 8657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:51.194183 8657 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:51.194226 8657 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:51.204253 8657 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:40889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:36509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:36509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:51.205479 8657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:51.206732 8657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:51.218178 8662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:51.218114 8663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:51.218152 8665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:51.219061 8657 server_base.cc:1034] running on GCE node
I20260619 02:55:51.219787 8657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:51.221128 8657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:51.222332 8657 hybrid_clock.cc:648] HybridClock initialized: now 1781837751222276 us; error 42 us; skew 500 ppm
I20260619 02:55:51.222719 8657 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:51 dist-test-slave-9mr0 krb5kdc[8644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837751, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:51.225644 8657 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:55:51.227123 8657 webserver.cc:460] Webserver started at http://127.18.134.62:40745/ using document root <none> and password file <none>
I20260619 02:55:51.227762 8657 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:51.227861 8657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:51.228099 8657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:51.231001 8657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "ee76dc0f61de4cfaa6a40a5cf4b29d4c"
format_stamp: "Formatted at 2026-06-19 02:55:51 on dist-test-slave-9mr0"
server_key: "998ed498d54b76b4b51f4247a3d55ffe"
server_key_iv: "01f3b209dad3e0e8576ea19e5862f891"
server_key_version: "encryptionkey@0"
I20260619 02:55:51.231698 8657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "ee76dc0f61de4cfaa6a40a5cf4b29d4c"
format_stamp: "Formatted at 2026-06-19 02:55:51 on dist-test-slave-9mr0"
server_key: "998ed498d54b76b4b51f4247a3d55ffe"
server_key_iv: "01f3b209dad3e0e8576ea19e5862f891"
server_key_version: "encryptionkey@0"
I20260619 02:55:51.236660 8657 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260619 02:55:51.240016 8672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:51.241328 8657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:55:51.241493 8657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "ee76dc0f61de4cfaa6a40a5cf4b29d4c"
format_stamp: "Formatted at 2026-06-19 02:55:51 on dist-test-slave-9mr0"
server_key: "998ed498d54b76b4b51f4247a3d55ffe"
server_key_iv: "01f3b209dad3e0e8576ea19e5862f891"
server_key_version: "encryptionkey@0"
I20260619 02:55:51.241668 8657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:51.260941 8657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:51.265017 8657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:51.265282 8657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:51.282382 8657 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:36509
I20260619 02:55:51.282406 8723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:36509 every 8 connection(s)
I20260619 02:55:51.283746 8657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:55:51.287384 8724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:51.288993 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 8657
I20260619 02:55:51.289160 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:55:51.289580 18968 external_mini_cluster.cc:1428] Setting key b3a4feb2ff615c9e9f35686d89ff75d4
I20260619 02:55:51.299503 8724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee76dc0f61de4cfaa6a40a5cf4b29d4c: Bootstrap starting.
I20260619 02:55:51.301718 8727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:51.291297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37907 (local address 127.18.134.62:36509)
0619 02:55:51.292109 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:51.292120 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:51.292161 (+ 41us) server_negotiation.cc:407] Connection header received
0619 02:55:51.293723 (+ 1562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:51.293744 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:51.294160 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:51.294500 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:51.295052 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:51.296265 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:51.297559 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:51.298205 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:51.298364 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:51.299550 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:51.299557 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:51.299571 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:51.300111 (+ 540us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:51.300185 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:51.300214 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:51.300311 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:51.300542 (+ 231us) server_negotiation.cc:299] Negotiation successful
0619 02:55:51.300782 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":324,"threads_started":1}
I20260619 02:55:51.302799 8724 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee76dc0f61de4cfaa6a40a5cf4b29d4c: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:51.303853 8724 log.cc:824] T 00000000000000000000000000000000 P ee76dc0f61de4cfaa6a40a5cf4b29d4c: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:51.305284 18968 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:55:51.306479 8724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee76dc0f61de4cfaa6a40a5cf4b29d4c: No bootstrap required, opened a new log
I20260619 02:55:51.314364 8724 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee76dc0f61de4cfaa6a40a5cf4b29d4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee76dc0f61de4cfaa6a40a5cf4b29d4c" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 36509 } }
I20260619 02:55:51.314783 8724 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee76dc0f61de4cfaa6a40a5cf4b29d4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:51.314955 8724 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee76dc0f61de4cfaa6a40a5cf4b29d4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee76dc0f61de4cfaa6a40a5cf4b29d4c, State: Initialized, Role: FOLLOWER
I20260619 02:55:51.315141 8727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:51.309335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37125 (local address 127.18.134.62:36509)
0619 02:55:51.309499 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:51.309505 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:51.309865 (+ 360us) server_negotiation.cc:407] Connection header received
0619 02:55:51.310039 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:51.310043 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:51.310104 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:51.310186 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:51.310693 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:51.311446 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:51.312748 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:51.313214 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:51.313340 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:51.314374 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:51.314378 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:51.314382 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:51.314399 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:51.314435 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:51.314449 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:51.314520 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:51.314676 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 02:55:51.314766 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 02:55:51.315568 8724 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee76dc0f61de4cfaa6a40a5cf4b29d4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee76dc0f61de4cfaa6a40a5cf4b29d4c" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 36509 } }
I20260619 02:55:51.315815 8724 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee76dc0f61de4cfaa6a40a5cf4b29d4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:55:51.315949 8724 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee76dc0f61de4cfaa6a40a5cf4b29d4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:55:51.316114 8724 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee76dc0f61de4cfaa6a40a5cf4b29d4c [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:51.318348 8724 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee76dc0f61de4cfaa6a40a5cf4b29d4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee76dc0f61de4cfaa6a40a5cf4b29d4c" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 36509 } }
I20260619 02:55:51.318871 8724 leader_election.cc:302] T 00000000000000000000000000000000 P ee76dc0f61de4cfaa6a40a5cf4b29d4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee76dc0f61de4cfaa6a40a5cf4b29d4c; no voters:
I20260619 02:55:51.319399 8724 leader_election.cc:288] T 00000000000000000000000000000000 P ee76dc0f61de4cfaa6a40a5cf4b29d4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:55:51.319511 8734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee76dc0f61de4cfaa6a40a5cf4b29d4c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:51.320210 8734 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee76dc0f61de4cfaa6a40a5cf4b29d4c [term 1 LEADER]: Becoming Leader. State: Replica: ee76dc0f61de4cfaa6a40a5cf4b29d4c, State: Running, Role: LEADER
I20260619 02:55:51.320730 8734 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee76dc0f61de4cfaa6a40a5cf4b29d4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee76dc0f61de4cfaa6a40a5cf4b29d4c" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 36509 } }
I20260619 02:55:51.321467 8724 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee76dc0f61de4cfaa6a40a5cf4b29d4c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:55:51.323154 8736 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee76dc0f61de4cfaa6a40a5cf4b29d4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee76dc0f61de4cfaa6a40a5cf4b29d4c. Latest consensus state: current_term: 1 leader_uuid: "ee76dc0f61de4cfaa6a40a5cf4b29d4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee76dc0f61de4cfaa6a40a5cf4b29d4c" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 36509 } } }
I20260619 02:55:51.323642 8736 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee76dc0f61de4cfaa6a40a5cf4b29d4c [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:51.323587 8735 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee76dc0f61de4cfaa6a40a5cf4b29d4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee76dc0f61de4cfaa6a40a5cf4b29d4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee76dc0f61de4cfaa6a40a5cf4b29d4c" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 36509 } } }
I20260619 02:55:51.323945 8735 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee76dc0f61de4cfaa6a40a5cf4b29d4c [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:51.327446 8741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:55:51.334414 8741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:55:51.344089 8741 catalog_manager.cc:1347] Generated new cluster ID: ffd5ede1ebf1419c88ae8f16aec6a53c
I20260619 02:55:51.344221 8741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:55:51.365860 8741 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:55:51.367210 8741 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:55:51.375222 8741 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee76dc0f61de4cfaa6a40a5cf4b29d4c: Generated new TSK 0
I20260619 02:55:51.376036 8741 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:55:51.390285 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 8657
W20260619 02:55:51.391366 8719 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37907
W20260619 02:55:51.392011 18968 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 8657: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 02:55:51.392131 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 8657
2026-06-19T02:55:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1781 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:55:51 dist-test-slave-9mr0 krb5kdc[8755](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:55:51 dist-test-slave-9mr0 krb5kdc[8755](info): set up 2 sockets
Jun 19 02:55:51 dist-test-slave-9mr0 krb5kdc[8755](info): commencing operation
krb5kdc: starting...
W20260619 02:55:52.406862 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.983s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:55:52 dist-test-slave-9mr0 krb5kdc[8755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:55:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:55:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:55:52.642159 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:36833
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:45855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:36833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:52.959743 8768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:52.960057 8768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:52.960199 8768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:52.967525 8768 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:52.967665 8768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:52.967744 8768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:52.967828 8768 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:52.967885 8768 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:52.977845 8768 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:45855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:36833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:36833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:52.979187 8768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:52.980515 8768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:52.992264 8773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:52.992249 8776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:52.992307 8774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:52.993117 8768 server_base.cc:1034] running on GCE node
I20260619 02:55:52.993877 8768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:52.995322 8768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:52.996536 8768 hybrid_clock.cc:648] HybridClock initialized: now 1781837752996490 us; error 41 us; skew 500 ppm
I20260619 02:55:52.996871 8768 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:53 dist-test-slave-9mr0 krb5kdc[8755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837753, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:53.008641 8768 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:55:53.010195 8768 webserver.cc:460] Webserver started at http://127.18.134.62:41521/ using document root <none> and password file <none>
I20260619 02:55:53.010934 8768 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:53.011044 8768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:53.011310 8768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:53.014338 8768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "00e1c4604a4f439d831369a6cc7c0461"
format_stamp: "Formatted at 2026-06-19 02:55:53 on dist-test-slave-9mr0"
server_key: "d86af015be9e9e27c315397e07cb3b6a"
server_key_iv: "55cf4f07f9d1ced9d505a74f7b37c838"
server_key_version: "encryptionkey@0"
I20260619 02:55:53.015147 8768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "00e1c4604a4f439d831369a6cc7c0461"
format_stamp: "Formatted at 2026-06-19 02:55:53 on dist-test-slave-9mr0"
server_key: "d86af015be9e9e27c315397e07cb3b6a"
server_key_iv: "55cf4f07f9d1ced9d505a74f7b37c838"
server_key_version: "encryptionkey@0"
I20260619 02:55:53.020399 8768 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260619 02:55:53.023943 8783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:53.025471 8768 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:55:53.025637 8768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "00e1c4604a4f439d831369a6cc7c0461"
format_stamp: "Formatted at 2026-06-19 02:55:53 on dist-test-slave-9mr0"
server_key: "d86af015be9e9e27c315397e07cb3b6a"
server_key_iv: "55cf4f07f9d1ced9d505a74f7b37c838"
server_key_version: "encryptionkey@0"
I20260619 02:55:53.025843 8768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:53.031670 8768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:53.035835 8768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:53.036089 8768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:53.053475 8768 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:36833
I20260619 02:55:53.053498 8834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:36833 every 8 connection(s)
I20260619 02:55:53.054955 8768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:55:53.058852 8835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:53.064699 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 8768
I20260619 02:55:53.064885 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:55:53.065284 18968 external_mini_cluster.cc:1428] Setting key f240da3f94b4b40de93f13542de11140
I20260619 02:55:53.071619 8835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00e1c4604a4f439d831369a6cc7c0461: Bootstrap starting.
I20260619 02:55:53.073988 8838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:53.067101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41373 (local address 127.18.134.62:36833)
0619 02:55:53.068016 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:53.068029 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:53.068092 (+ 63us) server_negotiation.cc:407] Connection header received
0619 02:55:53.070226 (+ 2134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:53.070255 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:53.070715 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:53.071292 (+ 577us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:53.072382 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:53.072388 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:53.072403 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:53.072417 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:53.072474 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:53.072494 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:53.072699 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:53.072843 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 02:55:53.073041 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":430,"threads_started":1}
I20260619 02:55:53.075217 8835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00e1c4604a4f439d831369a6cc7c0461: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:53.076349 8835 log.cc:824] T 00000000000000000000000000000000 P 00e1c4604a4f439d831369a6cc7c0461: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:53.078102 18968 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:55:53.079476 8835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00e1c4604a4f439d831369a6cc7c0461: No bootstrap required, opened a new log
I20260619 02:55:53.084983 8838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:53.082487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54639 (local address 127.18.134.62:36833)
0619 02:55:53.082736 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:53.082743 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:53.083164 (+ 421us) server_negotiation.cc:407] Connection header received
0619 02:55:53.083289 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:53.083295 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:53.083359 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:53.083467 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:53.084355 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:53.084361 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:53.084366 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 02:55:53.084383 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:53.084434 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 02:55:53.084454 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:53.084573 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:53.084615 (+ 42us) server_negotiation.cc:299] Negotiation successful
0619 02:55:53.084675 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 02:55:53.088001 8835 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00e1c4604a4f439d831369a6cc7c0461 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00e1c4604a4f439d831369a6cc7c0461" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 36833 } }
I20260619 02:55:53.088356 8835 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00e1c4604a4f439d831369a6cc7c0461 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 02:55:53.088430 8835 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00e1c4604a4f439d831369a6cc7c0461 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00e1c4604a4f439d831369a6cc7c0461, State: Initialized, Role: FOLLOWER
I20260619 02:55:53.088920 8835 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00e1c4604a4f439d831369a6cc7c0461 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00e1c4604a4f439d831369a6cc7c0461" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 36833 } }
I20260619 02:55:53.089123 8835 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00e1c4604a4f439d831369a6cc7c0461 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 02:55:53.089215 8835 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00e1c4604a4f439d831369a6cc7c0461 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 02:55:53.089354 8835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00e1c4604a4f439d831369a6cc7c0461 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:53.091648 8835 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00e1c4604a4f439d831369a6cc7c0461 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00e1c4604a4f439d831369a6cc7c0461" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 36833 } }
I20260619 02:55:53.092101 8835 leader_election.cc:302] T 00000000000000000000000000000000 P 00e1c4604a4f439d831369a6cc7c0461 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00e1c4604a4f439d831369a6cc7c0461; no voters:
I20260619 02:55:53.092620 8835 leader_election.cc:288] T 00000000000000000000000000000000 P 00e1c4604a4f439d831369a6cc7c0461 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 02:55:53.092751 8845 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00e1c4604a4f439d831369a6cc7c0461 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:53.093374 8845 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00e1c4604a4f439d831369a6cc7c0461 [term 1 LEADER]: Becoming Leader. State: Replica: 00e1c4604a4f439d831369a6cc7c0461, State: Running, Role: LEADER
I20260619 02:55:53.093981 8845 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00e1c4604a4f439d831369a6cc7c0461 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00e1c4604a4f439d831369a6cc7c0461" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 36833 } }
I20260619 02:55:53.094534 8835 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00e1c4604a4f439d831369a6cc7c0461 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:55:53.096714 8846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00e1c4604a4f439d831369a6cc7c0461 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00e1c4604a4f439d831369a6cc7c0461" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00e1c4604a4f439d831369a6cc7c0461" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 36833 } } }
I20260619 02:55:53.097127 8846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00e1c4604a4f439d831369a6cc7c0461 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:53.098083 8854 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:55:53.099501 8847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00e1c4604a4f439d831369a6cc7c0461 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00e1c4604a4f439d831369a6cc7c0461. Latest consensus state: current_term: 1 leader_uuid: "00e1c4604a4f439d831369a6cc7c0461" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00e1c4604a4f439d831369a6cc7c0461" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 36833 } } }
I20260619 02:55:53.100332 8847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00e1c4604a4f439d831369a6cc7c0461 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:53.105721 8854 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:55:53.114504 8854 catalog_manager.cc:1347] Generated new cluster ID: 68f068c76dac4f08a7a345e5e087704c
I20260619 02:55:53.114661 8854 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:55:53.126616 8854 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:55:53.127092 8854 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:55:53.136065 8854 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00e1c4604a4f439d831369a6cc7c0461: Generated new TSK 0
I20260619 02:55:53.137046 8854 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:55:53.151285 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 8768
W20260619 02:55:53.152290 8830 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41373
W20260619 02:55:53.152951 18968 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 8768: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 02:55:53.153105 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 8768
2026-06-19T02:55:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1760 ms)
[----------] 18 tests from AuthTokenIssuingTest (32577 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:55:53 dist-test-slave-9mr0 krb5kdc[8866](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:55:53 dist-test-slave-9mr0 krb5kdc[8866](info): set up 2 sockets
Jun 19 02:55:53 dist-test-slave-9mr0 krb5kdc[8866](info): commencing operation
krb5kdc: starting...
W20260619 02:55:54.171026 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:55:54 dist-test-slave-9mr0 krb5kdc[8866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:55:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:55:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:55:54.415678 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:37747
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:44399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:37747,127.18.134.61:34681,127.18.134.60:33231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:54.730718 8879 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:55:54.731128 8879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:54.731269 8879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:54.731381 8879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:54.738862 8879 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:54.738965 8879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:54.739063 8879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:54.739146 8879 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:54.739243 8879 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:54.749433 8879 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:44399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:37747,127.18.134.61:34681,127.18.134.60:33231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:37747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:54.750806 8879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:54.752152 8879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:54.764058 8885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:54.764060 8887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:54.764873 8884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:54.764887 8879 server_base.cc:1034] running on GCE node
I20260619 02:55:54.765767 8879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:54.767292 8879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:54.768508 8879 hybrid_clock.cc:648] HybridClock initialized: now 1781837754768445 us; error 41 us; skew 500 ppm
I20260619 02:55:54.768832 8879 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:54 dist-test-slave-9mr0 krb5kdc[8866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837754, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:54.771895 8879 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:55:54.773252 8879 webserver.cc:460] Webserver started at http://127.18.134.62:34577/ using document root <none> and password file <none>
I20260619 02:55:54.773937 8879 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:54.774037 8879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:54.774348 8879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:54.777349 8879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "58a4ddd5a1d148f2a298e792149faaf9"
format_stamp: "Formatted at 2026-06-19 02:55:54 on dist-test-slave-9mr0"
server_key: "9bc16717dcfcddf851bd58162b1d0523"
server_key_iv: "ec6b39194436d847f1aac328b3804f11"
server_key_version: "encryptionkey@0"
I20260619 02:55:54.778123 8879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "58a4ddd5a1d148f2a298e792149faaf9"
format_stamp: "Formatted at 2026-06-19 02:55:54 on dist-test-slave-9mr0"
server_key: "9bc16717dcfcddf851bd58162b1d0523"
server_key_iv: "ec6b39194436d847f1aac328b3804f11"
server_key_version: "encryptionkey@0"
I20260619 02:55:54.783203 8879 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260619 02:55:54.786651 8894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:54.787909 8879 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:55:54.788064 8879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "58a4ddd5a1d148f2a298e792149faaf9"
format_stamp: "Formatted at 2026-06-19 02:55:54 on dist-test-slave-9mr0"
server_key: "9bc16717dcfcddf851bd58162b1d0523"
server_key_iv: "ec6b39194436d847f1aac328b3804f11"
server_key_version: "encryptionkey@0"
I20260619 02:55:54.788260 8879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:54.804204 8879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:54.808454 8879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:54.808713 8879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:54.827061 8879 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:37747
I20260619 02:55:54.827097 8945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:37747 every 8 connection(s)
I20260619 02:55:54.828408 8879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:55:54.832125 8946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:54.838696 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 8879
I20260619 02:55:54.838910 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:55:54.838783 8946 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37747 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:55:54.839713 18968 external_mini_cluster.cc:1428] Setting key b1eb4d3df6d6f7d27b97723c01372f09
Jun 19 02:55:54 dist-test-slave-9mr0 krb5kdc[8866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837754, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:55:54.862146 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:54.842593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:37747 (local address 127.0.0.1:58324)
0619 02:55:54.843835 (+ 1242us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:54.843883 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:54.845595 (+ 1712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:54.846493 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:54.846498 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:54.847132 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:54.847468 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:54.847485 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:54.849128 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:54.849133 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:54.850209 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:54.850221 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:54.851360 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:54.851365 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:54.851498 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:54.852119 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:55:54.852140 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:54.854414 (+ 2274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:54.858136 (+ 3722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:54.858147 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:54.858156 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:54.858565 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:54.859023 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:54.859026 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:54.859028 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:54.859180 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:54.860150 (+ 970us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:54.860156 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:54.860355 (+ 199us) client_negotiation.cc:769] Sending connection context
0619 02:55:54.860565 (+ 210us) client_negotiation.cc:240] Negotiation successful
0619 02:55:54.860947 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":951,"spinlock_wait_cycles":67456,"thread_start_us":731,"threads_started":1}
I20260619 02:55:54.862182 8948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:54.842593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58324 (local address 127.18.134.62:37747)
0619 02:55:54.843449 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:54.843463 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:54.844020 (+ 557us) server_negotiation.cc:407] Connection header received
0619 02:55:54.845960 (+ 1940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:54.845967 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:54.846254 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:54.846458 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:54.847689 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:54.848988 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:54.850345 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:54.851145 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:54.851323 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:54.854671 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:54.854681 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:54.854696 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:54.854743 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:54.858012 (+ 3269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:54.858711 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:54.858720 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:54.858724 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:54.858922 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:54.859274 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:54.859278 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:54.859281 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:54.859998 (+ 717us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:54.860197 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:54.860752 (+ 555us) server_negotiation.cc:299] Negotiation successful
0619 02:55:54.861133 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":74,"server-negotiator.queue_time_us":596,"thread_start_us":380,"threads_started":1}
WARNING: no policy specified for kudu/127.18.134.61@KRBTEST.COM; defaulting to no policy
I20260619 02:55:54.865240 8946 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.61" port: 34681 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.61@KRBTEST.COM" created.
I20260619 02:55:54.866425 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:54.866001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:34681 (local address 127.0.0.1:51664)
0619 02:55:54.866174 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:54.866288 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:34681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260619 02:55:54.866830 8896 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.134.61:34681: connect: Connection refused (error 111)
I20260619 02:55:54.868321 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:54.867967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:34681 (local address 127.0.0.1:51668)
0619 02:55:54.868105 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:54.868174 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:34681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260619 02:55:54.868496 8946 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.61:34681: Network error: Client connection negotiation failed: client connection to 127.18.134.61:34681: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.61.keytab.
Entry for principal kudu/127.18.134.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.61.keytab.
I20260619 02:55:54.894155 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.61:34681
--webserver_interface=127.18.134.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.61.keytab
--principal=kudu/127.18.134.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:44399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:37747,127.18.134.61:34681,127.18.134.60:33231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:55:54.917835 8946 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.61" port: 34681 } attempt: 1
I20260619 02:55:54.919091 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:54.918556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:34681 (local address 127.0.0.1:51680)
0619 02:55:54.918790 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:54.918932 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:34681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260619 02:55:54.920171 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:54.919790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:34681 (local address 127.0.0.1:51692)
0619 02:55:54.919957 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:54.920029 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:34681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260619 02:55:54.920382 8946 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.61:34681: Network error: Client connection negotiation failed: client connection to 127.18.134.61:34681: connect: Connection refused (error 111)
I20260619 02:55:54.988687 8946 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.61" port: 34681 } attempt: 2
I20260619 02:55:54.990020 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:54.989484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:34681 (local address 127.0.0.1:51694)
0619 02:55:54.989776 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:54.989879 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:34681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260619 02:55:54.991083 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:54.990740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:34681 (local address 127.0.0.1:51706)
0619 02:55:54.990882 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:54.990954 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:34681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260619 02:55:54.991322 8946 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.61:34681: Network error: Client connection negotiation failed: client connection to 127.18.134.61:34681: connect: Connection refused (error 111)
I20260619 02:55:55.082578 8946 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.61" port: 34681 } attempt: 3
I20260619 02:55:55.083890 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.083433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:34681 (local address 127.0.0.1:51718)
0619 02:55:55.083636 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.083731 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:34681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260619 02:55:55.084872 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.084508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:34681 (local address 127.0.0.1:51724)
0619 02:55:55.084655 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.084727 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:34681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 02:55:55.085065 8946 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.61:34681: Network error: Client connection negotiation failed: client connection to 127.18.134.61:34681: connect: Connection refused (error 111)
W20260619 02:55:55.224423 8953 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:55:55.224778 8953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:55.224870 8953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:55.224982 8953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260619 02:55:55.228302 8946 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.61" port: 34681 } attempt: 4
I20260619 02:55:55.229435 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.228905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:34681 (local address 127.0.0.1:51730)
0619 02:55:55.229211 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.229302 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:34681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260619 02:55:55.230399 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.230045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:34681 (local address 127.0.0.1:51744)
0619 02:55:55.230199 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.230279 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:34681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260619 02:55:55.230566 8946 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.61:34681: Network error: Client connection negotiation failed: client connection to 127.18.134.61:34681: connect: Connection refused (error 111)
W20260619 02:55:55.232611 8953 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:55.232709 8953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:55.232805 8953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:55.232870 8953 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:55.232954 8953 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:55.243041 8953 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:44399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:37747,127.18.134.61:34681,127.18.134.60:33231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.61.keytab
--principal=kudu/127.18.134.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.61:34681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.134.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:55.244321 8953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:55.245585 8953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:55.256691 8959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:55.256932 8958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:55.256691 8961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:55.257498 8953 server_base.cc:1034] running on GCE node
I20260619 02:55:55.258263 8953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:55.259627 8953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:55.260828 8953 hybrid_clock.cc:648] HybridClock initialized: now 1781837755260760 us; error 50 us; skew 500 ppm
I20260619 02:55:55.261128 8953 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:55 dist-test-slave-9mr0 krb5kdc[8866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837755, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:55.264179 8953 init.cc:377] Logged in from keytab as kudu/127.18.134.61@KRBTEST.COM (short username kudu)
I20260619 02:55:55.265641 8953 webserver.cc:460] Webserver started at http://127.18.134.61:44461/ using document root <none> and password file <none>
I20260619 02:55:55.266312 8953 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:55.266395 8953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:55.266680 8953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:55.269520 8953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-1/data/instance:
uuid: "69946ef409c54c36a599c5777697d248"
format_stamp: "Formatted at 2026-06-19 02:55:55 on dist-test-slave-9mr0"
server_key: "b0710ec4103f857a09ec14360cf93c44"
server_key_iv: "c5f1e7b7832f72673c8510bebdca270a"
server_key_version: "encryptionkey@0"
I20260619 02:55:55.270260 8953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-1/wal/instance:
uuid: "69946ef409c54c36a599c5777697d248"
format_stamp: "Formatted at 2026-06-19 02:55:55 on dist-test-slave-9mr0"
server_key: "b0710ec4103f857a09ec14360cf93c44"
server_key_iv: "c5f1e7b7832f72673c8510bebdca270a"
server_key_version: "encryptionkey@0"
I20260619 02:55:55.275249 8953 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260619 02:55:55.278496 8968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:55.279788 8953 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:55:55.279934 8953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-1/wal
uuid: "69946ef409c54c36a599c5777697d248"
format_stamp: "Formatted at 2026-06-19 02:55:55 on dist-test-slave-9mr0"
server_key: "b0710ec4103f857a09ec14360cf93c44"
server_key_iv: "c5f1e7b7832f72673c8510bebdca270a"
server_key_version: "encryptionkey@0"
I20260619 02:55:55.280118 8953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:55.302742 8953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:55.306413 8953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:55.306669 8953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:55.322813 8953 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.61:34681
I20260619 02:55:55.322831 9019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.61:34681 every 8 connection(s)
I20260619 02:55:55.324096 8953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-1/data/info.pb
I20260619 02:55:55.327140 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 8953
I20260619 02:55:55.327306 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-1/wal/instance
I20260619 02:55:55.327682 9020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:55.327716 18968 external_mini_cluster.cc:1428] Setting key 9a5b24ee3a15af5023c63e1c26d3166e
I20260619 02:55:55.333988 9020 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37747 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:55:55 dist-test-slave-9mr0 krb5kdc[8866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837755, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.61@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.134.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.60@KRBTEST.COM" created.
I20260619 02:55:55.354444 8948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.336638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58340 (local address 127.18.134.62:37747)
0619 02:55:55.336844 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:55.336848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:55.338106 (+ 1258us) server_negotiation.cc:407] Connection header received
0619 02:55:55.340047 (+ 1941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:55.340053 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:55.340135 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:55.340225 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:55.341835 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.342528 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.344171 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.344598 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.344712 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:55.348260 (+ 3548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:55.348267 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:55.348272 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:55.348310 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:55.350912 (+ 2602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:55.351626 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:55.351633 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:55.351637 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:55.351788 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:55.352302 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:55.352308 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:55.352311 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:55.352680 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:55.352821 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:55.353879 (+ 1058us) server_negotiation.cc:299] Negotiation successful
0619 02:55:55.354119 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 02:55:55.356650 9023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.337200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:37747 (local address 127.0.0.1:58340)
0619 02:55:55.337931 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.337977 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:55.339751 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:55.340517 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:55.340526 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:55.341059 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:55.341645 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:55.341663 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.342715 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.342719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:55.344020 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:55.344032 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.344820 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.344827 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:55.345049 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:55.345747 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:55:55.345776 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:55.348039 (+ 2263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:55.351039 (+ 3000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:55.351048 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:55.351067 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:55.351492 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:55.351932 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:55.351936 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:55.351939 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:55.352170 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:55.352917 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:55.352924 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:55.353408 (+ 484us) client_negotiation.cc:769] Sending connection context
0619 02:55:55.354688 (+ 1280us) client_negotiation.cc:240] Negotiation successful
0619 02:55:55.355266 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":294,"threads_started":1}
I20260619 02:55:55.359858 9020 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.61" port: 34681 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:55:55 dist-test-slave-9mr0 krb5kdc[8866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837755, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.61@KRBTEST.COM for kudu/127.18.134.61@KRBTEST.COM
I20260619 02:55:55.379377 9023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.360658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:34681 (local address 127.0.0.1:51746)
0619 02:55:55.360837 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.360856 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:55.360971 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:55.364080 (+ 3109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:55.364085 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:55.364543 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:55.364726 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:55.364737 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.366761 (+ 2024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.366766 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:55.367994 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:55.368008 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.369045 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.369051 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:55.369242 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:55.369917 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:55:55.369941 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:55.372574 (+ 2633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:55.376369 (+ 3795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:55.376375 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:55.376378 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:55.376886 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:55.377341 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:55.377346 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:55.377350 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:55.377529 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:55.378364 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:55.378369 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:55.378524 (+ 155us) client_negotiation.cc:769] Sending connection context
0619 02:55:55.378712 (+ 188us) client_negotiation.cc:240] Negotiation successful
0619 02:55:55.378991 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 02:55:55.380383 9025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.362702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51746 (local address 127.18.134.61:34681)
0619 02:55:55.363477 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:55.363486 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:55.363521 (+ 35us) server_negotiation.cc:407] Connection header received
0619 02:55:55.363583 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:55.363592 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:55.363894 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:55.364060 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:55.364919 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.366597 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.368145 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.368882 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.368983 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:55.372828 (+ 3845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:55.372845 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:55.372865 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:55.372915 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:55.376224 (+ 3309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:55.377031 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:55.377041 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:55.377045 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:55.377214 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:55.377651 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:55.377657 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:55.377661 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:55.378193 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:55.378417 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:55.379737 (+ 1320us) server_negotiation.cc:299] Negotiation successful
0619 02:55:55.379982 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":411,"threads_started":1}
I20260619 02:55:55.382887 9020 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:55:55.384202 9023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.383676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:33231 (local address 127.0.0.1:50302)
0619 02:55:55.383876 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.384031 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260619 02:55:55.384433 8971 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
I20260619 02:55:55.386036 9023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.385707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:33231 (local address 127.0.0.1:50316)
0619 02:55:55.385851 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.385930 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260619 02:55:55.386233 9020 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.60:33231: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.60.keytab.
Entry for principal kudu/127.18.134.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.60.keytab.
I20260619 02:55:55.392338 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.60:33231
--webserver_interface=127.18.134.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.60.keytab
--principal=kudu/127.18.134.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:44399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:37747,127.18.134.61:34681,127.18.134.60:33231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:55:55.435572 9020 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } attempt: 1
I20260619 02:55:55.436748 9023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.436306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:33231 (local address 127.0.0.1:50328)
0619 02:55:55.436493 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.436604 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260619 02:55:55.437664 9023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.437329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:33231 (local address 127.0.0.1:50344)
0619 02:55:55.437481 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.437556 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260619 02:55:55.437857 9020 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.60:33231: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
I20260619 02:55:55.506069 9020 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } attempt: 2
I20260619 02:55:55.507218 9023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.506790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:33231 (local address 127.0.0.1:50346)
0619 02:55:55.506978 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.507072 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260619 02:55:55.508181 9023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.507854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:33231 (local address 127.0.0.1:50360)
0619 02:55:55.507997 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.508068 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260619 02:55:55.508375 9020 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.60:33231: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
I20260619 02:55:55.529852 8946 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.61" port: 34681 } attempt: 5
Jun 19 02:55:55 dist-test-slave-9mr0 krb5kdc[8866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837754, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for kudu/127.18.134.61@KRBTEST.COM
I20260619 02:55:55.543668 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.530623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:34681 (local address 127.0.0.1:51762)
0619 02:55:55.530878 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.530896 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:55.531009 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:55.531370 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:55.531373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:55.531730 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:55.531922 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:55.531930 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.533274 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.533282 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:55.534103 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:55.534114 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.535098 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.535102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:55.535259 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:55.535857 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:55:55.535876 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:55.537912 (+ 2036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:55.541475 (+ 3563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:55.541479 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:55.541482 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:55.541855 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:55.542206 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:55.542209 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:55.542211 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:55.542347 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:55.542948 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:55.542952 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:55.543091 (+ 139us) client_negotiation.cc:769] Sending connection context
0619 02:55:55.543201 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 02:55:55.543420 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260619 02:55:55.543896 9025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.530760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51762 (local address 127.18.134.61:34681)
0619 02:55:55.531003 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:55.531011 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:55.531037 (+ 26us) server_negotiation.cc:407] Connection header received
0619 02:55:55.531129 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:55.531135 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:55.531239 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:55.531356 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:55.532069 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.533162 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.534251 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.534948 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.535077 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:55.538110 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:55.538117 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:55.538122 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:55.538162 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:55.541336 (+ 3174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:55.541964 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:55.541969 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:55.541973 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:55.542115 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:55.542440 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:55.542445 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:55.542448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:55.542849 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:55.542992 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:55.543334 (+ 342us) server_negotiation.cc:299] Negotiation successful
0619 02:55:55.543566 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260619 02:55:55.544896 8946 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:55:55.545845 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.545473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:33231 (local address 127.0.0.1:50364)
0619 02:55:55.545629 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.545704 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260619 02:55:55.546728 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.546355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:33231 (local address 127.0.0.1:50376)
0619 02:55:55.546486 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.546584 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260619 02:55:55.546922 8946 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.60:33231: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
I20260619 02:55:55.598181 8946 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } attempt: 1
I20260619 02:55:55.599326 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.598881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:33231 (local address 127.0.0.1:50378)
0619 02:55:55.599071 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.599159 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260619 02:55:55.599555 9020 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } attempt: 3
I20260619 02:55:55.600466 9023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.600052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:33231 (local address 127.0.0.1:50386)
0619 02:55:55.600209 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.600296 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260619 02:55:55.600513 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.600102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:33231 (local address 127.0.0.1:50398)
0619 02:55:55.600313 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.600394 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260619 02:55:55.600696 8946 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.60:33231: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
I20260619 02:55:55.601598 9023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.601119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:33231 (local address 127.0.0.1:50408)
0619 02:55:55.601352 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.601449 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
W20260619 02:55:55.601807 9020 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.60:33231: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
I20260619 02:55:55.668896 8946 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } attempt: 2
I20260619 02:55:55.669984 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.669521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:33231 (local address 127.0.0.1:50412)
0619 02:55:55.669736 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.669829 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260619 02:55:55.671041 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.670721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:33231 (local address 127.0.0.1:50420)
0619 02:55:55.670862 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.670935 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260619 02:55:55.671240 8946 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.60:33231: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
W20260619 02:55:55.720654 9027 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:55:55.720983 9027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:55.721060 9027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:55.721141 9027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:55.728374 9027 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:55.728467 9027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:55.728523 9027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:55.728572 9027 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:55.728624 9027 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:55.738415 9027 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:44399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:37747,127.18.134.61:34681,127.18.134.60:33231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.60.keytab
--principal=kudu/127.18.134.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.60:33231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.134.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:55.739688 9027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:55.740948 9027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:55:55.745030 9020 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } attempt: 4
I20260619 02:55:55.746402 9023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.745744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:33231 (local address 127.0.0.1:50424)
0619 02:55:55.746042 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.746164 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260619 02:55:55.747639 9023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.747204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:33231 (local address 127.0.0.1:50438)
0619 02:55:55.747399 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.747496 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260619 02:55:55.747864 9020 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.60:33231: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
W20260619 02:55:55.752671 9033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:55.752704 9032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:55.752629 9035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:55.753539 9027 server_base.cc:1034] running on GCE node
I20260619 02:55:55.754297 9027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:55.755715 9027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:55.756929 9027 hybrid_clock.cc:648] HybridClock initialized: now 1781837755756860 us; error 47 us; skew 500 ppm
I20260619 02:55:55.757244 9027 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:55 dist-test-slave-9mr0 krb5kdc[8866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837755, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:55.760358 9027 init.cc:377] Logged in from keytab as kudu/127.18.134.60@KRBTEST.COM (short username kudu)
I20260619 02:55:55.761693 9027 webserver.cc:460] Webserver started at http://127.18.134.60:40983/ using document root <none> and password file <none>
I20260619 02:55:55.762349 9027 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:55.762436 9027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:55.762735 9027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:55.766029 9027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-2/data/instance:
uuid: "fdbf9867498a44a0b8dc716764cee965"
format_stamp: "Formatted at 2026-06-19 02:55:55 on dist-test-slave-9mr0"
server_key: "551b9a7c7b6702dd8b97f2d5974f9610"
server_key_iv: "0d3a3f86b5927c696ea222b44e297c46"
server_key_version: "encryptionkey@0"
I20260619 02:55:55.767045 9027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-2/wal/instance:
uuid: "fdbf9867498a44a0b8dc716764cee965"
format_stamp: "Formatted at 2026-06-19 02:55:55 on dist-test-slave-9mr0"
server_key: "551b9a7c7b6702dd8b97f2d5974f9610"
server_key_iv: "0d3a3f86b5927c696ea222b44e297c46"
server_key_version: "encryptionkey@0"
I20260619 02:55:55.772526 9027 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:55:55.775964 9042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:55.777398 9027 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 02:55:55.777437 8946 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } attempt: 3
I20260619 02:55:55.777572 9027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-2/wal
uuid: "fdbf9867498a44a0b8dc716764cee965"
format_stamp: "Formatted at 2026-06-19 02:55:55 on dist-test-slave-9mr0"
server_key: "551b9a7c7b6702dd8b97f2d5974f9610"
server_key_iv: "0d3a3f86b5927c696ea222b44e297c46"
server_key_version: "encryptionkey@0"
I20260619 02:55:55.777814 9027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:55.778537 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.777976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:33231 (local address 127.0.0.1:50446)
0619 02:55:55.778219 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.778337 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260619 02:55:55.779621 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.779170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:33231 (local address 127.0.0.1:50448)
0619 02:55:55.779408 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.779496 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
W20260619 02:55:55.779874 8946 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.60:33231: Network error: Client connection negotiation failed: client connection to 127.18.134.60:33231: connect: Connection refused (error 111)
I20260619 02:55:55.794266 9027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:55.798488 9027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:55.798756 9027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:55.816072 9027 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.60:33231
I20260619 02:55:55.816112 9093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.60:33231 every 8 connection(s)
I20260619 02:55:55.817410 9027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-2/data/info.pb
I20260619 02:55:55.821058 9094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:55.821331 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 9027
I20260619 02:55:55.821494 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781837410515220-18968-0/minicluster-data/master-2/wal/instance
I20260619 02:55:55.821907 18968 external_mini_cluster.cc:1428] Setting key 7f31b056514d28f7a1bdd8ffbd65bc3a
I20260619 02:55:55.830399 9094 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37747 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:55:55 dist-test-slave-9mr0 krb5kdc[8866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.60@KRBTEST.COM
Jun 19 02:55:55 dist-test-slave-9mr0 krb5kdc[8866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837755, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.60@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:55:55.844957 9097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.823687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50458 (local address 127.18.134.60:33231)
0619 02:55:55.824478 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:55.824490 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:55.824529 (+ 39us) server_negotiation.cc:407] Connection header received
0619 02:55:55.826040 (+ 1511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:55.826060 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:55.826554 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:55.827045 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:55.828809 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.830718 (+ 1909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.832513 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.833539 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.833735 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:55.838193 (+ 4458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:55.838201 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:55.838211 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:55.838250 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:55.841028 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:55.841674 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:55.841683 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:55.841691 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:55.841844 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:55.842174 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:55.842179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:55.842183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:55.842823 (+ 640us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:55.843091 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:55.843504 (+ 413us) server_negotiation.cc:299] Negotiation successful
0619 02:55:55.843914 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":352,"threads_started":1}
I20260619 02:55:55.847548 9098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.833244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:37747 (local address 127.0.0.1:58346)
0619 02:55:55.834024 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.834064 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:55.834270 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:55.834663 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:55.834671 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:55.835450 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:55.835757 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:55.835768 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.837057 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.837063 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:55.838120 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:55.838133 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.838875 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.838882 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:55.839051 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:55.839602 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:55:55.839618 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:55.841583 (+ 1965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:55.845047 (+ 3464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:55.845058 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:55.845065 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:55.845500 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:55.845851 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:55.845856 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:55.845860 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:55.845978 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:55.846492 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:55.846499 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:55.846712 (+ 213us) client_negotiation.cc:769] Sending connection context
0619 02:55:55.846900 (+ 188us) client_negotiation.cc:240] Negotiation successful
0619 02:55:55.847181 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":372,"threads_started":1}
I20260619 02:55:55.847978 8948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.833178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58346 (local address 127.18.134.62:37747)
0619 02:55:55.833361 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:55.833367 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:55.834152 (+ 785us) server_negotiation.cc:407] Connection header received
0619 02:55:55.834396 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:55.834401 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:55.834463 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:55.834550 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:55.835921 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.836912 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.838261 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.838703 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.838810 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:55.841758 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:55.841763 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:55.841766 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:55.841798 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:55.844930 (+ 3132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:55.845615 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:55.845619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:55.845622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:55.845748 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:55.846075 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:55.846078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:55.846080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:55.846353 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:55.846500 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:55.847020 (+ 520us) server_negotiation.cc:299] Negotiation successful
0619 02:55:55.847613 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 02:55:55.849265 9094 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.61" port: 34681 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:55:55 dist-test-slave-9mr0 krb5kdc[8866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837755, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.60@KRBTEST.COM for kudu/127.18.134.61@KRBTEST.COM
Jun 19 02:55:55 dist-test-slave-9mr0 krb5kdc[8866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:55:55.861718 9098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.849900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:34681 (local address 127.0.0.1:51778)
0619 02:55:55.850049 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.850066 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:55.850148 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:55.850459 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:55.850462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:55.850813 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:55.850949 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:55.850957 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.851898 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.851903 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:55.852959 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:55.852973 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.853702 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.853708 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:55.853906 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:55.854331 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:55:55.854353 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:55.856541 (+ 2188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:55.859443 (+ 2902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:55.859448 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:55.859452 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:55.859880 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:55.860189 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:55.860194 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:55.860197 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:55.860367 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:55.860888 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:55.860891 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:55.861053 (+ 162us) client_negotiation.cc:769] Sending connection context
0619 02:55:55.861195 (+ 142us) client_negotiation.cc:240] Negotiation successful
0619 02:55:55.861408 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 02:55:55.861754 9025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.850007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51778 (local address 127.18.134.61:34681)
0619 02:55:55.850178 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:55.850184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:55.850201 (+ 17us) server_negotiation.cc:407] Connection header received
0619 02:55:55.850255 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:55.850260 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:55.850352 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:55.850451 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:55.851075 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.851769 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.853106 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.853565 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.853677 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:55.856714 (+ 3037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:55.856720 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:55.856725 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:55.856776 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:55.859321 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:55.859976 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:55.859979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:55.859981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:55.860096 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:55.860465 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:55.860469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:55.860472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:55.860779 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:55.860940 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:55.861301 (+ 361us) server_negotiation.cc:299] Negotiation successful
0619 02:55:55.861508 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:55:55.862782 9094 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:55:55.862972 8948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.847727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58348 (local address 127.18.134.62:37747)
0619 02:55:55.848194 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:55.848200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:55.848507 (+ 307us) server_negotiation.cc:407] Connection header received
0619 02:55:55.848642 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:55.848648 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:55.848718 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:55.848811 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:55.850184 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.851127 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.852501 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.852904 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.852995 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:55.857529 (+ 4534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:55.857535 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:55.857540 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:55.857575 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:55.860259 (+ 2684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:55.860961 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:55.860966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:55.860970 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:55.861108 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:55.861461 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:55.861466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:55.861469 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:55.861811 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:55.862022 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:55.862439 (+ 417us) server_negotiation.cc:299] Negotiation successful
0619 02:55:55.862663 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":455,"threads_started":1}
Jun 19 02:55:55 dist-test-slave-9mr0 krb5kdc[8866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837755, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.60@KRBTEST.COM for kudu/127.18.134.60@KRBTEST.COM
Jun 19 02:55:55 dist-test-slave-9mr0 krb5kdc[8866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.61@KRBTEST.COM
I20260619 02:55:55.876169 9098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.863417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:33231 (local address 127.0.0.1:50462)
0619 02:55:55.863566 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.863586 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:55.863669 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:55.863948 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:55.863952 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:55.864311 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:55.864467 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:55.864478 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.865634 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.865638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:55.866892 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:55.866905 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.867734 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.867738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:55.867899 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:55.868437 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:55:55.868458 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:55.871037 (+ 2579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:55.873981 (+ 2944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:55.873987 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:55.873991 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:55.874316 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:55.874624 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:55.874662 (+ 38us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:55.874666 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:55.874796 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:55.875259 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:55.875263 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:55.875396 (+ 133us) client_negotiation.cc:769] Sending connection context
0619 02:55:55.875507 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 02:55:55.875790 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 02:55:55.876247 9097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.863593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50462 (local address 127.18.134.60:33231)
0619 02:55:55.863733 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:55.863738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:55.863753 (+ 15us) server_negotiation.cc:407] Connection header received
0619 02:55:55.863798 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:55.863802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:55.863849 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:55.863918 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:55.864603 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.865507 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.867015 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.867579 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.867705 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:55.871211 (+ 3506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:55.871218 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:55.871223 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:55.871260 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:55.873870 (+ 2610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:55.874414 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:55.874419 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:55.874422 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:55.874521 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:55.874888 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:55.874893 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:55.874896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:55.875157 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:55.875286 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:55.875615 (+ 329us) server_negotiation.cc:299] Negotiation successful
0619 02:55:55.875953 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":121,"server-negotiator.queue_time_us":49}
I20260619 02:55:55.878726 9025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.864447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51784 (local address 127.18.134.61:34681)
0619 02:55:55.864602 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:55.864607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:55.864621 (+ 14us) server_negotiation.cc:407] Connection header received
0619 02:55:55.864665 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:55.864669 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:55.864732 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:55.864871 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:55.865946 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.867030 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.868457 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.869049 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.869191 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:55.873538 (+ 4347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:55.873543 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:55.873546 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:55.873576 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:55.876074 (+ 2498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:55.876868 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:55.876873 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:55.876876 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:55.877035 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:55.877420 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:55.877423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:55.877425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:55.877695 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:55.877956 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:55.878166 (+ 210us) server_negotiation.cc:299] Negotiation successful
0619 02:55:55.878391 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 02:55:55.882177 9094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdbf9867498a44a0b8dc716764cee965: Bootstrap starting.
I20260619 02:55:55.883306 18968 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:55:55.886042 9094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdbf9867498a44a0b8dc716764cee965: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:55.887128 9094 log.cc:824] T 00000000000000000000000000000000 P fdbf9867498a44a0b8dc716764cee965: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:55.886518 8921 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "58a4ddd5a1d148f2a298e792149faaf9"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58348
I20260619 02:55:55.889780 9094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdbf9867498a44a0b8dc716764cee965: No bootstrap required, opened a new log
I20260619 02:55:55.896875 9094 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdbf9867498a44a0b8dc716764cee965 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4ddd5a1d148f2a298e792149faaf9" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37747 } } peers { permanent_uuid: "69946ef409c54c36a599c5777697d248" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 34681 } } peers { permanent_uuid: "fdbf9867498a44a0b8dc716764cee965" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } }
I20260619 02:55:55.897202 9094 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdbf9867498a44a0b8dc716764cee965 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdbf9867498a44a0b8dc716764cee965, State: Initialized, Role: FOLLOWER
I20260619 02:55:55.897642 9094 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdbf9867498a44a0b8dc716764cee965 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4ddd5a1d148f2a298e792149faaf9" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37747 } } peers { permanent_uuid: "69946ef409c54c36a599c5777697d248" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 34681 } } peers { permanent_uuid: "fdbf9867498a44a0b8dc716764cee965" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } }
I20260619 02:55:55.898484 9101 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdbf9867498a44a0b8dc716764cee965 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4ddd5a1d148f2a298e792149faaf9" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37747 } } peers { permanent_uuid: "69946ef409c54c36a599c5777697d248" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 34681 } } peers { permanent_uuid: "fdbf9867498a44a0b8dc716764cee965" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } } }
I20260619 02:55:55.898819 9101 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdbf9867498a44a0b8dc716764cee965 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:55:55.899565 9094 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdbf9867498a44a0b8dc716764cee965 [sys.catalog]: configured and running, proceeding with master startup.
W20260619 02:55:55.907972 9112 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fdbf9867498a44a0b8dc716764cee965: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:55:55.908079 9112 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:55:55.906608 9027 master_runner.cc:186] Error getting master registration for 127.18.134.62:37747: OK, instance_id { permanent_uuid: "58a4ddd5a1d148f2a298e792149faaf9" instance_seqno: 1781837754822042 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:55:55.917469 9027 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 02:55:55.957086 8946 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } attempt: 4
Jun 19 02:55:55 dist-test-slave-9mr0 krb5kdc[8866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837754, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for kudu/127.18.134.60@KRBTEST.COM
I20260619 02:55:55.972867 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.957870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:33231 (local address 127.0.0.1:50476)
0619 02:55:55.958121 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:55.958146 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:55.958311 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:55.958731 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:55.958737 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:55.959312 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:55.959545 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:55.959562 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.961026 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.961041 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:55.962564 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:55.962582 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.963601 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.963609 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:55.963844 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:55.964569 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:55:55.964597 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:55.967285 (+ 2688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:55.970422 (+ 3137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:55.970428 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:55.970432 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:55.970867 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:55.971216 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:55.971221 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:55.971224 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:55.971362 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:55.971904 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:55.971909 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:55.972070 (+ 161us) client_negotiation.cc:769] Sending connection context
0619 02:55:55.972184 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 02:55:55.972459 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260619 02:55:55.972872 9097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:55.957906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50476 (local address 127.18.134.60:33231)
0619 02:55:55.958122 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:55.958127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:55.958238 (+ 111us) server_negotiation.cc:407] Connection header received
0619 02:55:55.958459 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:55.958465 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:55.958556 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:55.958687 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:55.959717 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.960858 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.962734 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:55.963445 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:55.963567 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:55.967483 (+ 3916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:55.967490 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:55.967495 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:55.967535 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:55.970291 (+ 2756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:55.970977 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:55.970983 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:55.970987 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:55.971112 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:55.971472 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:55.971477 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:55.971481 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:55.971790 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:55.971947 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:55.972333 (+ 386us) server_negotiation.cc:299] Negotiation successful
0619 02:55:55.972539 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 02:55:55.977247 8946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9: Bootstrap starting.
I20260619 02:55:55.980386 8946 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:55.981432 8946 log.cc:824] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:55.984565 8946 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9: No bootstrap required, opened a new log
I20260619 02:55:55.989621 8946 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4ddd5a1d148f2a298e792149faaf9" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37747 } } peers { permanent_uuid: "69946ef409c54c36a599c5777697d248" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 34681 } } peers { permanent_uuid: "fdbf9867498a44a0b8dc716764cee965" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } }
I20260619 02:55:55.989923 8946 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58a4ddd5a1d148f2a298e792149faaf9, State: Initialized, Role: FOLLOWER
I20260619 02:55:55.990336 8946 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4ddd5a1d148f2a298e792149faaf9" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37747 } } peers { permanent_uuid: "69946ef409c54c36a599c5777697d248" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 34681 } } peers { permanent_uuid: "fdbf9867498a44a0b8dc716764cee965" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } }
I20260619 02:55:55.991173 9115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4ddd5a1d148f2a298e792149faaf9" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37747 } } peers { permanent_uuid: "69946ef409c54c36a599c5777697d248" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 34681 } } peers { permanent_uuid: "fdbf9867498a44a0b8dc716764cee965" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } } }
I20260619 02:55:55.991569 9115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:55:55.992383 8946 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:55:55.992589 8921 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 02:55:55.992910 8921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:55.996564 8921 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4ddd5a1d148f2a298e792149faaf9" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37747 } } peers { permanent_uuid: "69946ef409c54c36a599c5777697d248" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 34681 } } peers { permanent_uuid: "fdbf9867498a44a0b8dc716764cee965" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } }
I20260619 02:55:55.999482 9069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "58a4ddd5a1d148f2a298e792149faaf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fdbf9867498a44a0b8dc716764cee965"
I20260619 02:55:55.999825 8921 leader_election.cc:288] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9 [CANDIDATE]: Term 1 election: Requested vote from peers 69946ef409c54c36a599c5777697d248 (127.18.134.61:34681), fdbf9867498a44a0b8dc716764cee965 (127.18.134.60:33231)
I20260619 02:55:56.000001 9069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdbf9867498a44a0b8dc716764cee965 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:56.003578 9069 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fdbf9867498a44a0b8dc716764cee965 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58a4ddd5a1d148f2a298e792149faaf9 in term 1.
I20260619 02:55:56.004285 8897 leader_election.cc:302] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58a4ddd5a1d148f2a298e792149faaf9, fdbf9867498a44a0b8dc716764cee965; no voters:
I20260619 02:55:56.004750 9115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:55:56.010927 9115 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9 [term 1 LEADER]: Becoming Leader. State: Replica: 58a4ddd5a1d148f2a298e792149faaf9, State: Running, Role: LEADER
I20260619 02:55:56.011533 9115 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4ddd5a1d148f2a298e792149faaf9" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37747 } } peers { permanent_uuid: "69946ef409c54c36a599c5777697d248" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 34681 } } peers { permanent_uuid: "fdbf9867498a44a0b8dc716764cee965" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } }
I20260619 02:55:56.020684 9123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58a4ddd5a1d148f2a298e792149faaf9. Latest consensus state: current_term: 1 leader_uuid: "58a4ddd5a1d148f2a298e792149faaf9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4ddd5a1d148f2a298e792149faaf9" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37747 } } peers { permanent_uuid: "69946ef409c54c36a599c5777697d248" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 34681 } } peers { permanent_uuid: "fdbf9867498a44a0b8dc716764cee965" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } } }
I20260619 02:55:56.021188 9123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:56.022086 9131 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:55:56.039552 9131 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:55:56.047052 9020 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } attempt: 5
W20260619 02:55:56.050877 9048 tablet.cc:2369] T 00000000000000000000000000000000 P fdbf9867498a44a0b8dc716764cee965: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 02:55:56 dist-test-slave-9mr0 krb5kdc[8866](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837755, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.61@KRBTEST.COM for kudu/127.18.134.60@KRBTEST.COM
W20260619 02:55:56.061483 8900 tablet.cc:2369] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:55:56.068401 9023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:56.047655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:33231 (local address 127.0.0.1:50490)
0619 02:55:56.048030 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:56.048052 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:56.048174 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:56.049058 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:56.049063 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:56.049613 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:56.049835 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:56.049848 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.051378 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.051384 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:56.052875 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:56.052890 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.054424 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.054431 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:56.054693 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:56.055388 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:55:56.055411 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:56.058507 (+ 3096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:56.065489 (+ 6982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:56.065494 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:56.065498 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:56.066050 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:56.066499 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:56.066504 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:56.066508 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:56.066709 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:56.067390 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:56.067395 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:56.067553 (+ 158us) client_negotiation.cc:769] Sending connection context
0619 02:55:56.067725 (+ 172us) client_negotiation.cc:240] Negotiation successful
0619 02:55:56.068036 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258}
I20260619 02:55:56.069495 9097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:56.048393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50490 (local address 127.18.134.60:33231)
0619 02:55:56.048572 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.048577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.048608 (+ 31us) server_negotiation.cc:407] Connection header received
0619 02:55:56.048673 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.048679 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.048760 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.048855 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:56.050029 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.051205 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.053373 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.054085 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.054215 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:56.058855 (+ 4640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:56.058865 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:56.058870 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:56.058913 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:56.065287 (+ 6374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:56.066202 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:56.066208 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:56.066212 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:56.066371 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:56.066839 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:56.066844 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:56.066848 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:56.067192 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:56.067331 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:56.068942 (+ 1611us) server_negotiation.cc:299] Negotiation successful
0619 02:55:56.069177 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 02:55:56.039783 8879 master_runner.cc:186] Error getting master registration for 127.18.134.61:34681: OK, instance_id { permanent_uuid: "69946ef409c54c36a599c5777697d248" instance_seqno: 1781837755318149 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:55:56.075680 9020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69946ef409c54c36a599c5777697d248: Bootstrap starting.
I20260619 02:55:56.075920 8879 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 02:55:56.080508 9020 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69946ef409c54c36a599c5777697d248: Neither blocks nor log segments found. Creating new log.
I20260619 02:55:56.084867 9020 log.cc:824] T 00000000000000000000000000000000 P 69946ef409c54c36a599c5777697d248: Log is configured to *not* fsync() on all Append() calls
I20260619 02:55:56.094193 8948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:56.043585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58358 (local address 127.18.134.62:37747)
0619 02:55:56.043779 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.043785 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.062814 (+ 19029us) server_negotiation.cc:407] Connection header received
0619 02:55:56.063041 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.063049 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.063142 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.063245 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:56.073517 (+ 10272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.074586 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.077936 (+ 3350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.078670 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.078769 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:56.085867 (+ 7098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:56.085874 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:56.085879 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:56.085923 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:56.090222 (+ 4299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:56.091117 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:56.091124 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:56.091128 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:56.091354 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:56.092050 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:56.092055 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:56.092059 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:56.092449 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:56.092844 (+ 395us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:56.093480 (+ 636us) server_negotiation.cc:299] Negotiation successful
0619 02:55:56.093792 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 02:55:56.107833 9138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:56.063570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50506 (local address 127.18.134.60:33231)
0619 02:55:56.064232 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.064238 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.069432 (+ 5194us) server_negotiation.cc:407] Connection header received
0619 02:55:56.069489 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.069494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.069563 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.069655 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:56.080019 (+ 10364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.081209 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.085332 (+ 4123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.085988 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.086087 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:56.099906 (+ 13819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:56.099913 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:56.099918 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:56.099965 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:56.103495 (+ 3530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:56.104963 (+ 1468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:56.104968 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:56.104972 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:56.105151 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:56.105568 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:56.105573 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:56.105577 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:56.105936 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:56.106165 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:56.107155 (+ 990us) server_negotiation.cc:299] Negotiation successful
0619 02:55:56.107403 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":283,"threads_started":1}
I20260619 02:55:56.111507 9069 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fdbf9867498a44a0b8dc716764cee965 [term 1 FOLLOWER]: Refusing update from remote peer 58a4ddd5a1d148f2a298e792149faaf9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:55:56.112792 9123 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdbf9867498a44a0b8dc716764cee965" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:55:56.112978 9025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:56.044114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51800 (local address 127.18.134.61:34681)
0619 02:55:56.044294 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.044300 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.065504 (+ 21204us) server_negotiation.cc:407] Connection header received
0619 02:55:56.066861 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.066867 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.066946 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.067014 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:56.076921 (+ 9907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.078087 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.083140 (+ 5053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.083850 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.083950 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:56.098836 (+ 14886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:56.098844 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:56.098849 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:56.098897 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:56.103207 (+ 4310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:56.104298 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:56.104304 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:56.104308 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:56.104489 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:56.105177 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:56.105205 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:56.105209 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:56.105819 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:56.106192 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:56.107215 (+ 1023us) server_negotiation.cc:299] Negotiation successful
0619 02:55:56.107649 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 02:55:56.124435 9020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69946ef409c54c36a599c5777697d248: No bootstrap required, opened a new log
I20260619 02:55:56.137063 9020 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69946ef409c54c36a599c5777697d248 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4ddd5a1d148f2a298e792149faaf9" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37747 } } peers { permanent_uuid: "69946ef409c54c36a599c5777697d248" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 34681 } } peers { permanent_uuid: "fdbf9867498a44a0b8dc716764cee965" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } }
I20260619 02:55:56.137511 9020 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69946ef409c54c36a599c5777697d248 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69946ef409c54c36a599c5777697d248, State: Initialized, Role: FOLLOWER
I20260619 02:55:56.138077 9020 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69946ef409c54c36a599c5777697d248 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4ddd5a1d148f2a298e792149faaf9" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37747 } } peers { permanent_uuid: "69946ef409c54c36a599c5777697d248" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 34681 } } peers { permanent_uuid: "fdbf9867498a44a0b8dc716764cee965" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } }
I20260619 02:55:56.138971 9101 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdbf9867498a44a0b8dc716764cee965 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58a4ddd5a1d148f2a298e792149faaf9. Latest consensus state: current_term: 1 leader_uuid: "58a4ddd5a1d148f2a298e792149faaf9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4ddd5a1d148f2a298e792149faaf9" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37747 } } peers { permanent_uuid: "69946ef409c54c36a599c5777697d248" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 34681 } } peers { permanent_uuid: "fdbf9867498a44a0b8dc716764cee965" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } } }
I20260619 02:55:56.139321 9101 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdbf9867498a44a0b8dc716764cee965 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:55:56.146242 9139 mvcc.cc:205] Tried to move back new op lower bound from 7298407448956989440 to 7298407448629047296. Current Snapshot: MvccSnapshot[applied={T|T < 7298407448956989440}]
I20260619 02:55:56.160877 9143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69946ef409c54c36a599c5777697d248 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4ddd5a1d148f2a298e792149faaf9" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37747 } } peers { permanent_uuid: "69946ef409c54c36a599c5777697d248" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 34681 } } peers { permanent_uuid: "fdbf9867498a44a0b8dc716764cee965" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } } }
I20260619 02:55:56.164027 9131 catalog_manager.cc:1347] Generated new cluster ID: c722b654a54d418e9b8c9757e98029a5
I20260619 02:55:56.164165 9131 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:55:56.164352 9123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "58a4ddd5a1d148f2a298e792149faaf9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4ddd5a1d148f2a298e792149faaf9" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37747 } } peers { permanent_uuid: "69946ef409c54c36a599c5777697d248" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 34681 } } peers { permanent_uuid: "fdbf9867498a44a0b8dc716764cee965" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } } }
I20260619 02:55:56.164628 9123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:56.162369 8995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "58a4ddd5a1d148f2a298e792149faaf9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "69946ef409c54c36a599c5777697d248"
I20260619 02:55:56.163051 8994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69946ef409c54c36a599c5777697d248 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:55:56.170455 8994 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 69946ef409c54c36a599c5777697d248 [term 1 FOLLOWER]: Refusing update from remote peer 58a4ddd5a1d148f2a298e792149faaf9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:55:56.173812 8995 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 69946ef409c54c36a599c5777697d248 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58a4ddd5a1d148f2a298e792149faaf9 in term 1.
I20260619 02:55:56.175680 9115 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69946ef409c54c36a599c5777697d248" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:55:56.163182 9020 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69946ef409c54c36a599c5777697d248 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:55:56.177431 9143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69946ef409c54c36a599c5777697d248 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:55:56.177569 9101 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdbf9867498a44a0b8dc716764cee965 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "58a4ddd5a1d148f2a298e792149faaf9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4ddd5a1d148f2a298e792149faaf9" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37747 } } peers { permanent_uuid: "69946ef409c54c36a599c5777697d248" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 34681 } } peers { permanent_uuid: "fdbf9867498a44a0b8dc716764cee965" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } } }
I20260619 02:55:56.177824 9101 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdbf9867498a44a0b8dc716764cee965 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:55:56.198419 9131 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:55:56.200331 9131 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:55:56.224148 9131 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9: Generated new TSK 0
I20260619 02:55:56.225870 9131 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:55:56.235286 9143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69946ef409c54c36a599c5777697d248 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58a4ddd5a1d148f2a298e792149faaf9. Latest consensus state: current_term: 1 leader_uuid: "58a4ddd5a1d148f2a298e792149faaf9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4ddd5a1d148f2a298e792149faaf9" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37747 } } peers { permanent_uuid: "69946ef409c54c36a599c5777697d248" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 34681 } } peers { permanent_uuid: "fdbf9867498a44a0b8dc716764cee965" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } } }
I20260619 02:55:56.235616 9143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69946ef409c54c36a599c5777697d248 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:55:56.236194 9152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69946ef409c54c36a599c5777697d248 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "58a4ddd5a1d148f2a298e792149faaf9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4ddd5a1d148f2a298e792149faaf9" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37747 } } peers { permanent_uuid: "69946ef409c54c36a599c5777697d248" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 34681 } } peers { permanent_uuid: "fdbf9867498a44a0b8dc716764cee965" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } } }
I20260619 02:55:56.236673 9152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69946ef409c54c36a599c5777697d248 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:55:56.237897 9123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "58a4ddd5a1d148f2a298e792149faaf9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4ddd5a1d148f2a298e792149faaf9" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 37747 } } peers { permanent_uuid: "69946ef409c54c36a599c5777697d248" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 34681 } } peers { permanent_uuid: "fdbf9867498a44a0b8dc716764cee965" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 33231 } } }
I20260619 02:55:56.238181 9123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58a4ddd5a1d148f2a298e792149faaf9 [sys.catalog]: This master's current role is: LEADER
I20260619 02:55:56.299207 9162 catalog_manager.cc:1259] Loaded cluster ID: c722b654a54d418e9b8c9757e98029a5
I20260619 02:55:56.299369 9162 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 69946ef409c54c36a599c5777697d248: loading cluster ID for follower catalog manager: success
I20260619 02:55:56.311125 9162 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 69946ef409c54c36a599c5777697d248: acquiring CA information for follower catalog manager: success
I20260619 02:55:56.315969 9162 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 69946ef409c54c36a599c5777697d248: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 02:55:56.405572 8948 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:55:56.390688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58360 (local address 127.18.134.62:37747)
0619 02:55:56.390902 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.390908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.402951 (+ 12043us) server_negotiation.cc:407] Connection header received
0619 02:55:56.403519 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.403527 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.403618 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.403703 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:56.405313 (+ 1610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58360: BlockingRecv error: recv got EOF from 127.0.0.1:58360 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260619 02:55:56.408032 9025 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:55:56.390874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51802 (local address 127.18.134.61:34681)
0619 02:55:56.391065 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.391071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.406381 (+ 15310us) server_negotiation.cc:407] Connection header received
0619 02:55:56.406672 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.406678 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.406776 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.406856 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:56.407805 (+ 949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51802: BlockingRecv error: recv got EOF from 127.0.0.1:51802 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260619 02:55:56.410734 9138 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:55:56.408387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50512 (local address 127.18.134.60:33231)
0619 02:55:56.408578 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.408584 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.409075 (+ 491us) server_negotiation.cc:407] Connection header received
0619 02:55:56.409325 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.409331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.409408 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.409473 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:56.410467 (+ 994us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50512: BlockingRecv error: recv got EOF from 127.0.0.1:50512 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 02:55:56.463750 9025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:56.444952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51808 (local address 127.18.134.61:34681)
0619 02:55:56.445149 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.445156 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.455785 (+ 10629us) server_negotiation.cc:407] Connection header received
0619 02:55:56.456361 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.456368 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.456592 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.456748 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:55:56.457446 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.458548 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.460512 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.461182 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.461287 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:56.462085 (+ 798us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:55:56.462851 (+ 766us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:55:56.462975 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:56.463227 (+ 252us) server_negotiation.cc:299] Negotiation successful
0619 02:55:56.463336 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260619 02:55:56.506910 9138 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:55:56.496973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50526 (local address 127.18.134.60:33231)
0619 02:55:56.497178 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.497184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.504753 (+ 7569us) server_negotiation.cc:407] Connection header received
0619 02:55:56.505270 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.505277 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.505414 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.505492 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:56.506572 (+ 1080us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50526: BlockingRecv error: recv got EOF from 127.0.0.1:50526 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 02:55:56.542913 9025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:56.535394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51824 (local address 127.18.134.61:34681)
0619 02:55:56.535615 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.535621 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.536676 (+ 1055us) server_negotiation.cc:407] Connection header received
0619 02:55:56.536890 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.536897 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.537004 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.537108 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:55:56.537607 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.538822 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.540758 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.541429 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.541557 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:56.542056 (+ 499us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:55:56.542208 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:55:56.542306 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:56.542459 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 02:55:56.542542 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260619 02:55:56.552910 9138 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:55:56.550822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50536 (local address 127.18.134.60:33231)
0619 02:55:56.551006 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.551013 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.551509 (+ 496us) server_negotiation.cc:407] Connection header received
0619 02:55:56.551707 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.551713 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.551815 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.551917 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:56.552681 (+ 764us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50536: BlockingRecv error: recv got EOF from 127.0.0.1:50536 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 02:55:56.565320 9025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:56.558449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51832 (local address 127.18.134.61:34681)
0619 02:55:56.558687 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.558693 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.559154 (+ 461us) server_negotiation.cc:407] Connection header received
0619 02:55:56.559330 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.559336 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.559425 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.559519 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:55:56.559963 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.561096 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.563051 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.563814 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.563914 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:56.564497 (+ 583us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:55:56.564651 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:55:56.564744 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:56.564892 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 02:55:56.564983 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260619 02:55:56.574460 9138 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:55:56.572383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50540 (local address 127.18.134.60:33231)
0619 02:55:56.572576 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.572583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.573098 (+ 515us) server_negotiation.cc:407] Connection header received
0619 02:55:56.573250 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.573256 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.573362 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.573459 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:56.574230 (+ 771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50540: BlockingRecv error: recv got EOF from 127.0.0.1:50540 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:55:56.588814 9025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:56.581861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51840 (local address 127.18.134.61:34681)
0619 02:55:56.582074 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.582080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.582547 (+ 467us) server_negotiation.cc:407] Connection header received
0619 02:55:56.582849 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.582855 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.582948 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.583076 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:55:56.583538 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.584768 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.586803 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.587497 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.587655 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:56.588058 (+ 403us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:55:56.588193 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:55:56.588281 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:56.588425 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 02:55:56.588520 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
W20260619 02:55:56.597340 9138 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:55:56.595511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50550 (local address 127.18.134.60:33231)
0619 02:55:56.595759 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.595765 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.595930 (+ 165us) server_negotiation.cc:407] Connection header received
0619 02:55:56.596101 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.596106 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.596195 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.596307 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:56.597106 (+ 799us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50550: BlockingRecv error: recv got EOF from 127.0.0.1:50550 (error 108)
Metrics: {"server-negotiator.queue_time_us":131}
I20260619 02:55:56.618886 9025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:56.612463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51854 (local address 127.18.134.61:34681)
0619 02:55:56.612663 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.612670 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.613153 (+ 483us) server_negotiation.cc:407] Connection header received
0619 02:55:56.613335 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.613341 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.613425 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.613516 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:55:56.613935 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.615042 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.616933 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.617591 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.617709 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:56.618072 (+ 363us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:55:56.618210 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:55:56.618299 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:56.618434 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 02:55:56.618515 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260619 02:55:56.626406 9138 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:55:56.624455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50564 (local address 127.18.134.60:33231)
0619 02:55:56.624618 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.624623 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.625070 (+ 447us) server_negotiation.cc:407] Connection header received
0619 02:55:56.625254 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.625260 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.625334 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.625415 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:56.626178 (+ 763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50564: BlockingRecv error: recv got EOF from 127.0.0.1:50564 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 02:55:56.651150 9025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:56.645463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51860 (local address 127.18.134.61:34681)
0619 02:55:56.645658 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.645664 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.645981 (+ 317us) server_negotiation.cc:407] Connection header received
0619 02:55:56.646142 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.646148 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.646244 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.646359 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:55:56.646733 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.647928 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.649250 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.649886 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.650020 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:56.650412 (+ 392us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:55:56.650552 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:55:56.650682 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:56.650794 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 02:55:56.650888 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260619 02:55:56.658479 9138 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:55:56.656757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50568 (local address 127.18.134.60:33231)
0619 02:55:56.656960 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.656966 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.657177 (+ 211us) server_negotiation.cc:407] Connection header received
0619 02:55:56.657325 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.657330 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.657429 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.657538 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:56.658266 (+ 728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50568: BlockingRecv error: recv got EOF from 127.0.0.1:50568 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 02:55:56.699368 9025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:56.693610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51872 (local address 127.18.134.61:34681)
0619 02:55:56.693885 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.693892 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.694136 (+ 244us) server_negotiation.cc:407] Connection header received
0619 02:55:56.694291 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.694297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.694433 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.694567 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:55:56.694962 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.696077 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.697383 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.698082 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.698215 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:56.698573 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:55:56.698731 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:55:56.698853 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:56.698942 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 02:55:56.699025 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260619 02:55:56.706946 9138 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:55:56.705154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50580 (local address 127.18.134.60:33231)
0619 02:55:56.705345 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.705351 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.705646 (+ 295us) server_negotiation.cc:407] Connection header received
0619 02:55:56.705782 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.705787 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.705895 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.706000 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:56.706701 (+ 701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50580: BlockingRecv error: recv got EOF from 127.0.0.1:50580 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 02:55:56.781490 9025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:56.776393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51880 (local address 127.18.134.61:34681)
0619 02:55:56.776569 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.776575 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.776976 (+ 401us) server_negotiation.cc:407] Connection header received
0619 02:55:56.777191 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.777195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.777265 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.777347 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:55:56.777761 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.778514 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.779911 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.780329 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.780438 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:56.780828 (+ 390us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:55:56.780931 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:55:56.781005 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:56.781136 (+ 131us) server_negotiation.cc:299] Negotiation successful
0619 02:55:56.781216 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260619 02:55:56.788408 9138 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:55:56.786831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50594 (local address 127.18.134.60:33231)
0619 02:55:56.787031 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.787037 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.787327 (+ 290us) server_negotiation.cc:407] Connection header received
0619 02:55:56.787476 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.787481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.787555 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.787625 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:56.788204 (+ 579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50594: BlockingRecv error: recv got EOF from 127.0.0.1:50594 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 02:55:56.910058 9112 catalog_manager.cc:1259] Loaded cluster ID: c722b654a54d418e9b8c9757e98029a5
I20260619 02:55:56.910158 9112 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fdbf9867498a44a0b8dc716764cee965: loading cluster ID for follower catalog manager: success
I20260619 02:55:56.912997 9112 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fdbf9867498a44a0b8dc716764cee965: acquiring CA information for follower catalog manager: success
I20260619 02:55:56.914912 9112 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fdbf9867498a44a0b8dc716764cee965: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:55:56.925117 9025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:56.919546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51882 (local address 127.18.134.61:34681)
0619 02:55:56.919710 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.919715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.920079 (+ 364us) server_negotiation.cc:407] Connection header received
0619 02:55:56.920218 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.920223 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.920298 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.920390 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:55:56.920852 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.921619 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.923390 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.923857 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.923966 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:56.924423 (+ 457us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:55:56.924550 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:55:56.924631 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:56.924816 (+ 185us) server_negotiation.cc:299] Negotiation successful
0619 02:55:56.924888 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 02:55:56.940207 9138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:56.934589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50602 (local address 127.18.134.60:33231)
0619 02:55:56.934853 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.934860 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.935121 (+ 261us) server_negotiation.cc:407] Connection header received
0619 02:55:56.935333 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.935339 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.935503 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.935614 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:55:56.935994 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.936786 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.938064 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.938455 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.938563 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:56.939086 (+ 523us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:55:56.939606 (+ 520us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:55:56.939739 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:56.939852 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 02:55:56.939934 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260619 02:55:56.952879 9025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:56.946170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51890 (local address 127.18.134.61:34681)
0619 02:55:56.946376 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.946382 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.947133 (+ 751us) server_negotiation.cc:407] Connection header received
0619 02:55:56.947384 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.947390 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.947483 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.947562 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:55:56.948438 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.949518 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.951117 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.951563 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.951673 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:56.952171 (+ 498us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:55:56.952320 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:55:56.952393 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:56.952521 (+ 128us) server_negotiation.cc:299] Negotiation successful
0619 02:55:56.952603 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 02:55:56.954144 9138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:56.947387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50618 (local address 127.18.134.60:33231)
0619 02:55:56.947564 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.947569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.948408 (+ 839us) server_negotiation.cc:407] Connection header received
0619 02:55:56.948642 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.948647 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.948739 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.948827 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:55:56.949357 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.950290 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.952260 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.952875 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.952994 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:56.953498 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:55:56.953604 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:55:56.953678 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:56.953814 (+ 136us) server_negotiation.cc:299] Negotiation successful
0619 02:55:56.953890 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 02:55:56.954319 9266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:56.946007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58362 (local address 127.18.134.62:37747)
0619 02:55:56.946502 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:56.946509 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:56.947244 (+ 735us) server_negotiation.cc:407] Connection header received
0619 02:55:56.947528 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:56.947535 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:56.947779 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:56.947965 (+ 186us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:55:56.948731 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.949709 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.951868 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:56.952467 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:56.952583 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:56.953238 (+ 655us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:55:56.953650 (+ 412us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:55:56.953732 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:56.953952 (+ 220us) server_negotiation.cc:299] Negotiation successful
0619 02:55:56.954042 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":229,"threads_started":1}
I20260619 02:55:56.959141 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 8879
I20260619 02:55:56.960561 8941 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:57.068872 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 8879
I20260619 02:55:57.085726 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 8953
I20260619 02:55:57.086764 9015 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:57.191883 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 8953
I20260619 02:55:57.207659 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 9027
I20260619 02:55:57.208716 9089 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:55:57.312852 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 9027
2026-06-19T02:55:57Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4163 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 02:55:57 dist-test-slave-9mr0 krb5kdc[9275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 02:55:57 dist-test-slave-9mr0 krb5kdc[9275](info): set up 2 sockets
Jun 19 02:55:57 dist-test-slave-9mr0 krb5kdc[9275](info): commencing operation
krb5kdc: starting...
W20260619 02:55:58.334944 18968 mini_kdc.cc:121] Time spent starting KDC: real 0.987s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/krb5kdc/test-user.keytab.
Jun 19 02:55:58 dist-test-slave-9mr0 krb5kdc[9275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T02:55:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T02:55:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.134.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
Entry for principal kudu/127.18.134.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab.
I20260619 02:55:58.575402 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:35951
--webserver_interface=127.18.134.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:38033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:35951,127.18.134.61:35733,127.18.134.60:37421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 02:55:58.896528 9288 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:55:58.896886 9288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:58.896994 9288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:58.897112 9288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:58.904332 9288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:58.904433 9288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:58.904515 9288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:58.904580 9288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:58.904662 9288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:58.915040 9288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:38033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:35951,127.18.134.61:35733,127.18.134.60:37421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.62.keytab
--principal=kudu/127.18.134.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.62:35951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.134.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:58.916379 9288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:58.917773 9288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:58.929316 9293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:58.929352 9294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:58.929304 9296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:58.930589 9288 server_base.cc:1034] running on GCE node
I20260619 02:55:58.931388 9288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:58.932785 9288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:58.934003 9288 hybrid_clock.cc:648] HybridClock initialized: now 1781837758933937 us; error 43 us; skew 500 ppm
I20260619 02:55:58.934340 9288 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:58 dist-test-slave-9mr0 krb5kdc[9275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837758, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:58.941534 9288 init.cc:377] Logged in from keytab as kudu/127.18.134.62@KRBTEST.COM (short username kudu)
I20260619 02:55:58.942956 9288 webserver.cc:460] Webserver started at http://127.18.134.62:39507/ using document root <none> and password file <none>
I20260619 02:55:58.943622 9288 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:58.943707 9288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:58.943967 9288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:58.946903 9288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-0/data/instance:
uuid: "0777f73666074a3fb84a979853d92ee4"
format_stamp: "Formatted at 2026-06-19 02:55:58 on dist-test-slave-9mr0"
server_key: "df053aa386b6aae2cf5cd48a5ce5ae23"
server_key_iv: "cd0c3834c8685041868442745ba1be04"
server_key_version: "encryptionkey@0"
I20260619 02:55:58.947639 9288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-0/wal/instance:
uuid: "0777f73666074a3fb84a979853d92ee4"
format_stamp: "Formatted at 2026-06-19 02:55:58 on dist-test-slave-9mr0"
server_key: "df053aa386b6aae2cf5cd48a5ce5ae23"
server_key_iv: "cd0c3834c8685041868442745ba1be04"
server_key_version: "encryptionkey@0"
I20260619 02:55:58.952800 9288 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 02:55:58.956378 9303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:58.957803 9288 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 02:55:58.957966 9288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-0/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-0/wal
uuid: "0777f73666074a3fb84a979853d92ee4"
format_stamp: "Formatted at 2026-06-19 02:55:58 on dist-test-slave-9mr0"
server_key: "df053aa386b6aae2cf5cd48a5ce5ae23"
server_key_iv: "cd0c3834c8685041868442745ba1be04"
server_key_version: "encryptionkey@0"
I20260619 02:55:58.958170 9288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:58.975579 9288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:58.979602 9288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:58.979857 9288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:58.997256 9288 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.62:35951
I20260619 02:55:58.997263 9354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.62:35951 every 8 connection(s)
I20260619 02:55:58.998665 9288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-0/data/info.pb
I20260619 02:55:59.002502 9355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:59.008963 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 9288
I20260619 02:55:59.009145 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-0/wal/instance
I20260619 02:55:59.009558 18968 external_mini_cluster.cc:1428] Setting key f52f1089ac9c80c8e576fea076cf8409
I20260619 02:55:59.009367 9355 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35951 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:55:59 dist-test-slave-9mr0 krb5kdc[9275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837758, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.134.61@KRBTEST.COM; defaulting to no policy
I20260619 02:55:59.030869 9358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.012456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57592 (local address 127.18.134.62:35951)
0619 02:55:59.014037 (+ 1581us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:59.014045 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:59.014095 (+ 50us) server_negotiation.cc:407] Connection header received
0619 02:55:59.015603 (+ 1508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:59.015609 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:59.015843 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:59.016073 (+ 230us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:59.017315 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.018482 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.019841 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.020386 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.020549 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:59.023904 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:59.023912 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:59.023924 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:59.023952 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:59.026704 (+ 2752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:59.027565 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:59.027572 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:59.027574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:59.027729 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:59.028166 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:59.028169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:59.028171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:59.028663 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:59.028815 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:59.029490 (+ 675us) server_negotiation.cc:299] Negotiation successful
0619 02:55:59.029751 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1377,"thread_start_us":1186,"threads_started":1}
I20260619 02:55:59.030930 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.012455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35951 (local address 127.0.0.1:57592)
0619 02:55:59.013263 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.013311 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:59.015263 (+ 1952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:59.016093 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:59.016099 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:59.016802 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:59.017124 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:59.017145 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.018654 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.018660 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:59.019716 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:59.019727 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.020601 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.020617 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:59.020780 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:59.021357 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:55:59.021378 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:59.023698 (+ 2320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:59.026851 (+ 3153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:59.026864 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:59.026877 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:59.027423 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:59.027840 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:59.027845 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:59.027849 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:59.028066 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:59.028815 (+ 749us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:59.028824 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:59.029079 (+ 255us) client_negotiation.cc:769] Sending connection context
0619 02:55:59.029357 (+ 278us) client_negotiation.cc:240] Negotiation successful
0619 02:55:59.029949 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"mutex_wait_us":94,"thread_start_us":337,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.61@KRBTEST.COM" created.
I20260619 02:55:59.033812 9355 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35733 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:55:59.034883 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.034446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:35733 (local address 127.0.0.1:47734)
0619 02:55:59.034610 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.034762 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:35733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260619 02:55:59.035130 9304 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.134.61:35733: connect: Connection refused (error 111)
I20260619 02:55:59.036628 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.036287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:35733 (local address 127.0.0.1:47746)
0619 02:55:59.036423 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.036495 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:35733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 02:55:59.036840 9355 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.61:35733: Network error: Client connection negotiation failed: client connection to 127.18.134.61:35733: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.61.keytab.
Entry for principal kudu/127.18.134.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.61.keytab.
I20260619 02:55:59.062952 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.61:35733
--webserver_interface=127.18.134.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.61.keytab
--principal=kudu/127.18.134.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:38033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:35951,127.18.134.61:35733,127.18.134.60:37421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:55:59.086145 9355 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35733 } attempt: 1
I20260619 02:55:59.087340 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.086907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:35733 (local address 127.0.0.1:47750)
0619 02:55:59.087090 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.087207 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:35733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260619 02:55:59.088270 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.087928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:35733 (local address 127.0.0.1:47752)
0619 02:55:59.088050 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.088119 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:35733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260619 02:55:59.088438 9355 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.61:35733: Network error: Client connection negotiation failed: client connection to 127.18.134.61:35733: connect: Connection refused (error 111)
I20260619 02:55:59.156657 9355 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35733 } attempt: 2
I20260619 02:55:59.157886 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.157424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:35733 (local address 127.0.0.1:47762)
0619 02:55:59.157623 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.157734 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:35733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260619 02:55:59.158955 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.158571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:35733 (local address 127.0.0.1:47772)
0619 02:55:59.158733 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.158828 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:35733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260619 02:55:59.159149 9355 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.61:35733: Network error: Client connection negotiation failed: client connection to 127.18.134.61:35733: connect: Connection refused (error 111)
I20260619 02:55:59.250402 9355 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35733 } attempt: 3
I20260619 02:55:59.251597 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.251155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:35733 (local address 127.0.0.1:47788)
0619 02:55:59.251368 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.251463 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:35733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260619 02:55:59.252660 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.252293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:35733 (local address 127.0.0.1:47796)
0619 02:55:59.252444 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.252521 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:35733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260619 02:55:59.252840 9355 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.61:35733: Network error: Client connection negotiation failed: client connection to 127.18.134.61:35733: connect: Connection refused (error 111)
W20260619 02:55:59.379447 9362 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:55:59.379792 9362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:59.379896 9362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:59.380018 9362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:59.387290 9362 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:59.387387 9362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:59.387458 9362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:59.387532 9362 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:59.387621 9362 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:59.396061 9355 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35733 } attempt: 4
I20260619 02:55:59.397477 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.396821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:35733 (local address 127.0.0.1:47808)
0619 02:55:59.397219 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.397338 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:35733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182}
I20260619 02:55:59.398057 9362 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:38033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:35951,127.18.134.61:35733,127.18.134.60:37421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.61.keytab
--principal=kudu/127.18.134.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.61:35733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.134.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:59.398435 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.398069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:35733 (local address 127.0.0.1:47824)
0619 02:55:59.398212 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.398300 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.61:35733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260619 02:55:59.398604 9355 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.61:35733: Network error: Client connection negotiation failed: client connection to 127.18.134.61:35733: connect: Connection refused (error 111)
I20260619 02:55:59.399415 9362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:59.400734 9362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 02:55:59.411743 9367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:59.411809 9368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:59.411863 9370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:59.412875 9362 server_base.cc:1034] running on GCE node
I20260619 02:55:59.413597 9362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:59.414978 9362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:59.416214 9362 hybrid_clock.cc:648] HybridClock initialized: now 1781837759416137 us; error 46 us; skew 500 ppm
I20260619 02:55:59.416563 9362 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:59 dist-test-slave-9mr0 krb5kdc[9275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837759, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:59.419514 9362 init.cc:377] Logged in from keytab as kudu/127.18.134.61@KRBTEST.COM (short username kudu)
I20260619 02:55:59.421005 9362 webserver.cc:460] Webserver started at http://127.18.134.61:46593/ using document root <none> and password file <none>
I20260619 02:55:59.421645 9362 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:59.421727 9362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:59.422004 9362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:59.424912 9362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-1/data/instance:
uuid: "f3e10f76d2294e418cc52e37c44dcf56"
format_stamp: "Formatted at 2026-06-19 02:55:59 on dist-test-slave-9mr0"
server_key: "c4879add5e69c20f1d77c4043355ffce"
server_key_iv: "2addf9bb57ded7b18c12cd68a082fcb2"
server_key_version: "encryptionkey@0"
I20260619 02:55:59.425644 9362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-1/wal/instance:
uuid: "f3e10f76d2294e418cc52e37c44dcf56"
format_stamp: "Formatted at 2026-06-19 02:55:59 on dist-test-slave-9mr0"
server_key: "c4879add5e69c20f1d77c4043355ffce"
server_key_iv: "2addf9bb57ded7b18c12cd68a082fcb2"
server_key_version: "encryptionkey@0"
I20260619 02:55:59.430558 9362 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.007s
I20260619 02:55:59.433800 9377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:59.435132 9362 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260619 02:55:59.435286 9362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-1/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-1/wal
uuid: "f3e10f76d2294e418cc52e37c44dcf56"
format_stamp: "Formatted at 2026-06-19 02:55:59 on dist-test-slave-9mr0"
server_key: "c4879add5e69c20f1d77c4043355ffce"
server_key_iv: "2addf9bb57ded7b18c12cd68a082fcb2"
server_key_version: "encryptionkey@0"
I20260619 02:55:59.435475 9362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:59.453383 9362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:59.457196 9362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:59.457446 9362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:59.474483 9362 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.61:35733
I20260619 02:55:59.474503 9428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.61:35733 every 8 connection(s)
I20260619 02:55:59.475852 9362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-1/data/info.pb
I20260619 02:55:59.479513 9429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:59.485814 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 9362
I20260619 02:55:59.486012 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-1/wal/instance
I20260619 02:55:59.486538 18968 external_mini_cluster.cc:1428] Setting key eeadb0f77443e825375dee2e197fd5e4
I20260619 02:55:59.486173 9429 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35951 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:55:59 dist-test-slave-9mr0 krb5kdc[9275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837759, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.61@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:55:59.506599 9358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.489335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57606 (local address 127.18.134.62:35951)
0619 02:55:59.489549 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:59.489556 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:59.490730 (+ 1174us) server_negotiation.cc:407] Connection header received
0619 02:55:59.492616 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:59.492621 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:59.492707 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:59.492814 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:59.494262 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.494994 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.496531 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.496942 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.497052 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:59.500680 (+ 3628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:59.500685 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:59.500688 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:59.500724 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:59.503344 (+ 2620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:59.504231 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:59.504235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:59.504237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:59.504361 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:59.504864 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:59.504868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:59.504870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:59.505176 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:59.505303 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:59.506038 (+ 735us) server_negotiation.cc:299] Negotiation successful
0619 02:55:59.506284 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 02:55:59.507586 9431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.489762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35951 (local address 127.0.0.1:57606)
0619 02:55:59.490544 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.490596 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:59.492353 (+ 1757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:59.493000 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:59.493008 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:59.493528 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:59.494075 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:59.494092 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.495146 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.495150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:59.496382 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:59.496395 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.497157 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.497164 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:59.497386 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:59.498029 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:55:59.498050 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:59.500403 (+ 2353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:59.503486 (+ 3083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:59.503496 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:59.503514 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:59.504128 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:59.504455 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:59.504459 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:59.504463 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:59.504704 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:59.505308 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:59.505315 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:59.505659 (+ 344us) client_negotiation.cc:769] Sending connection context
0619 02:55:59.505917 (+ 258us) client_negotiation.cc:240] Negotiation successful
0619 02:55:59.506434 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":332,"threads_started":1}
WARNING: no policy specified for kudu/127.18.134.60@KRBTEST.COM; defaulting to no policy
I20260619 02:55:59.509405 9429 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35733 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.134.60@KRBTEST.COM" created.
Jun 19 02:55:59 dist-test-slave-9mr0 krb5kdc[9275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837759, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.61@KRBTEST.COM for kudu/127.18.134.61@KRBTEST.COM
I20260619 02:55:59.523190 9431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.510046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:35733 (local address 127.0.0.1:47828)
0619 02:55:59.510200 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.510219 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:59.510320 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:59.511360 (+ 1040us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:59.511364 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:59.511762 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:59.511886 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:59.511895 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.513208 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.513212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:59.514055 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:59.514066 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.514892 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.514899 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:59.515056 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:59.515580 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:55:59.515596 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:59.517683 (+ 2087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:59.520739 (+ 3056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:59.520745 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:59.520749 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:59.521184 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:59.521531 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:59.521535 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:59.521539 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:59.521703 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:59.522253 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:59.522257 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:59.522391 (+ 134us) client_negotiation.cc:769] Sending connection context
0619 02:55:59.522524 (+ 133us) client_negotiation.cc:240] Negotiation successful
0619 02:55:59.522837 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260619 02:55:59.523234 9433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.510328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47828 (local address 127.18.134.61:35733)
0619 02:55:59.510920 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:59.510926 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:59.510950 (+ 24us) server_negotiation.cc:407] Connection header received
0619 02:55:59.510996 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:59.511002 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:59.511212 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:59.511349 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:59.512040 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.513082 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.514182 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.514765 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.514871 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:59.517917 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:59.517932 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:59.517950 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:59.517986 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:59.520592 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:59.521307 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:59.521313 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:59.521316 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:59.521424 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:59.521795 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:59.521798 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:59.521800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:59.522119 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:59.522282 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:59.522710 (+ 428us) server_negotiation.cc:299] Negotiation successful
0619 02:55:59.522935 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":462,"thread_start_us":328,"threads_started":1}
I20260619 02:55:59.525565 9429 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:55:59.526767 9431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.526296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:37421 (local address 127.0.0.1:53738)
0619 02:55:59.526457 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.526570 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260619 02:55:59.527052 9380 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
I20260619 02:55:59.528491 9431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.528185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:37421 (local address 127.0.0.1:53754)
0619 02:55:59.528316 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.528395 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 02:55:59.528699 9429 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.60:37421: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.134.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.60.keytab.
Entry for principal kudu/127.18.134.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.60.keytab.
I20260619 02:55:59.542016 18968 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
/tmp/dist-test-taskUJ45gP/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.60:37421
--webserver_interface=127.18.134.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.60.keytab
--principal=kudu/127.18.134.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.134.20:38033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.134.62:35951,127.18.134.61:35733,127.18.134.60:37421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 02:55:59.578038 9429 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } attempt: 1
I20260619 02:55:59.579273 9431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.578799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:37421 (local address 127.0.0.1:53762)
0619 02:55:59.578999 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.579112 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260619 02:55:59.580247 9431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.579893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:37421 (local address 127.0.0.1:53778)
0619 02:55:59.580061 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.580139 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260619 02:55:59.580454 9429 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.60:37421: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
I20260619 02:55:59.648648 9429 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } attempt: 2
I20260619 02:55:59.649799 9431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.649345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:37421 (local address 127.0.0.1:53788)
0619 02:55:59.649526 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.649618 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260619 02:55:59.650767 9431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.650411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:37421 (local address 127.0.0.1:53798)
0619 02:55:59.650533 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.650604 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260619 02:55:59.650972 9429 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.60:37421: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
I20260619 02:55:59.697907 9355 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35733 } attempt: 5
Jun 19 02:55:59 dist-test-slave-9mr0 krb5kdc[9275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837758, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for kudu/127.18.134.61@KRBTEST.COM
I20260619 02:55:59.710798 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.698778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:35733 (local address 127.0.0.1:47842)
0619 02:55:59.698999 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.699021 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:59.699155 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:59.699581 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:59.699587 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:59.699998 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:59.700184 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:59.700194 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.701171 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.701176 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:59.702029 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:59.702042 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.702834 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.702840 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:59.703015 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:59.703597 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:55:59.703614 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:59.705553 (+ 1939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:59.708396 (+ 2843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:59.708403 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:59.708407 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:59.708790 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:59.709133 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:59.709138 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:59.709142 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:59.709298 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:59.709807 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:59.709812 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:59.709957 (+ 145us) client_negotiation.cc:769] Sending connection context
0619 02:55:59.710108 (+ 151us) client_negotiation.cc:240] Negotiation successful
0619 02:55:59.710399 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260619 02:55:59.710850 9433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.698871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47842 (local address 127.18.134.61:35733)
0619 02:55:59.699055 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:59.699060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:59.699082 (+ 22us) server_negotiation.cc:407] Connection header received
0619 02:55:59.699299 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:59.699303 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:59.699428 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:59.699520 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:59.700352 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.701037 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.702184 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.702664 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.702773 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:59.705753 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:59.705758 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:59.705762 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:59.705795 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:59.708261 (+ 2466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:59.708904 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:59.708908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:59.708910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:59.709028 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:59.709403 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:59.709409 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:59.709411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:59.709688 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:59.709818 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:59.710250 (+ 432us) server_negotiation.cc:299] Negotiation successful
0619 02:55:59.710469 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260619 02:55:59.711997 9355 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } has no permanent_uuid. Determining permanent_uuid...
I20260619 02:55:59.713160 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.712785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:37421 (local address 127.0.0.1:43826)
0619 02:55:59.712946 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.713026 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260619 02:55:59.714092 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.713777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:37421 (local address 127.0.0.1:43842)
0619 02:55:59.713920 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.713997 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260619 02:55:59.714288 9355 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.60:37421: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
I20260619 02:55:59.742177 9429 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } attempt: 3
I20260619 02:55:59.743270 9431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.742815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:37421 (local address 127.0.0.1:43846)
0619 02:55:59.743021 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.743118 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260619 02:55:59.744369 9431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.743957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:37421 (local address 127.0.0.1:43848)
0619 02:55:59.744187 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.744262 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
W20260619 02:55:59.744575 9429 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.60:37421: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
I20260619 02:55:59.765493 9355 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } attempt: 1
I20260619 02:55:59.766613 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.766158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:37421 (local address 127.0.0.1:43860)
0619 02:55:59.766344 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.766448 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260619 02:55:59.767647 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.767311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:37421 (local address 127.0.0.1:43868)
0619 02:55:59.767471 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.767541 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260619 02:55:59.767849 9355 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.60:37421: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
I20260619 02:55:59.836066 9355 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } attempt: 2
I20260619 02:55:59.837330 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.836792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:37421 (local address 127.0.0.1:43878)
0619 02:55:59.837059 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.837159 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260619 02:55:59.838366 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.838024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:37421 (local address 127.0.0.1:43886)
0619 02:55:59.838178 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.838262 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260619 02:55:59.838658 9355 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.60:37421: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
W20260619 02:55:59.863721 9436 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 02:55:59.864105 9436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 02:55:59.864198 9436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 02:55:59.864318 9436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 02:55:59.871454 9436 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 02:55:59.871555 9436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 02:55:59.871652 9436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 02:55:59.871718 9436 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 02:55:59.871799 9436 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 02:55:59.881479 9436 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.134.20:38033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.134.62:35951,127.18.134.61:35733,127.18.134.60:37421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/krb5kdc/kudu_127.18.134.60.keytab
--principal=kudu/127.18.134.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.134.60:37421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.134.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 01:54:22 UTC on ccycloud.quasar-vafwje.root.comops.site
build id 4454977
ASAN enabled
I20260619 02:55:59.882778 9436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 02:55:59.884020 9436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 02:55:59.887791 9429 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } attempt: 4
I20260619 02:55:59.889016 9431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.888437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:37421 (local address 127.0.0.1:43894)
0619 02:55:59.888696 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.888816 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260619 02:55:59.890239 9431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.889814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:37421 (local address 127.0.0.1:43902)
0619 02:55:59.890033 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.890131 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
W20260619 02:55:59.890436 9429 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.60:37421: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
W20260619 02:55:59.895344 9441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:59.895391 9444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 02:55:59.895416 9442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 02:55:59.896314 9436 server_base.cc:1034] running on GCE node
I20260619 02:55:59.897038 9436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 02:55:59.898372 9436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 02:55:59.899591 9436 hybrid_clock.cc:648] HybridClock initialized: now 1781837759899515 us; error 41 us; skew 500 ppm
I20260619 02:55:59.899911 9436 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 02:55:59 dist-test-slave-9mr0 krb5kdc[9275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837759, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 02:55:59.902956 9436 init.cc:377] Logged in from keytab as kudu/127.18.134.60@KRBTEST.COM (short username kudu)
I20260619 02:55:59.904279 9436 webserver.cc:460] Webserver started at http://127.18.134.60:46029/ using document root <none> and password file <none>
I20260619 02:55:59.904948 9436 fs_manager.cc:359] Metadata directory not provided
I20260619 02:55:59.905043 9436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 02:55:59.905330 9436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 02:55:59.908313 9436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-2/data/instance:
uuid: "ec2191bf86ab464da415dda416e87355"
format_stamp: "Formatted at 2026-06-19 02:55:59 on dist-test-slave-9mr0"
server_key: "eb2195dfb1732d6aaa7985d14ce913ec"
server_key_iv: "8f1572a5f758508f154e8993e9fc3b27"
server_key_version: "encryptionkey@0"
I20260619 02:55:59.909101 9436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-2/wal/instance:
uuid: "ec2191bf86ab464da415dda416e87355"
format_stamp: "Formatted at 2026-06-19 02:55:59 on dist-test-slave-9mr0"
server_key: "eb2195dfb1732d6aaa7985d14ce913ec"
server_key_iv: "8f1572a5f758508f154e8993e9fc3b27"
server_key_version: "encryptionkey@0"
I20260619 02:55:59.913951 9436 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 02:55:59.917223 9451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 02:55:59.918458 9436 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 02:55:59.918614 9436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-2/data,/tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-2/wal
uuid: "ec2191bf86ab464da415dda416e87355"
format_stamp: "Formatted at 2026-06-19 02:55:59 on dist-test-slave-9mr0"
server_key: "eb2195dfb1732d6aaa7985d14ce913ec"
server_key_iv: "8f1572a5f758508f154e8993e9fc3b27"
server_key_version: "encryptionkey@0"
I20260619 02:55:59.918857 9436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 02:55:59.937731 9436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 02:55:59.942126 9436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 02:55:59.942373 9436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 02:55:59.944887 9355 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } attempt: 3
I20260619 02:55:59.946039 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.945520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:37421 (local address 127.0.0.1:43906)
0619 02:55:59.945736 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.945843 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260619 02:55:59.947105 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.946612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:37421 (local address 127.0.0.1:43918)
0619 02:55:59.946903 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.946984 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186}
W20260619 02:55:59.947337 9355 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.134.60:37421: Network error: Client connection negotiation failed: client connection to 127.18.134.60:37421: connect: Connection refused (error 111)
I20260619 02:55:59.959430 9436 rpc_server.cc:307] RPC server started. Bound to: 127.18.134.60:37421
I20260619 02:55:59.959457 9502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.134.60:37421 every 8 connection(s)
I20260619 02:55:59.960729 9436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-2/data/info.pb
I20260619 02:55:59.964460 9503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 02:55:59.965857 18968 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu as pid 9436
I20260619 02:55:59.966043 18968 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskUJ45gP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781837410515220-18968-0/minicluster-data/master-2/wal/instance
I20260619 02:55:59.966573 18968 external_mini_cluster.cc:1428] Setting key c10bbff59b5907408053affb66c339c6
I20260619 02:55:59.973312 9503 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35951 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:55:59 dist-test-slave-9mr0 krb5kdc[9275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.60@KRBTEST.COM
Jun 19 02:55:59 dist-test-slave-9mr0 krb5kdc[9275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837759, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.60@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:55:59.989348 9506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.968466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43930 (local address 127.18.134.60:37421)
0619 02:55:59.969197 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:59.969209 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:59.969247 (+ 38us) server_negotiation.cc:407] Connection header received
0619 02:55:59.970900 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:59.970922 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:59.971326 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:59.971715 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:59.973124 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.974438 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.975764 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.976779 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.976937 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:59.981331 (+ 4394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:59.981342 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:59.981355 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:59.981405 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:59.984983 (+ 3578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:59.985816 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:59.985825 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:59.985831 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:59.985989 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:59.986371 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:59.986376 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:59.986378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:59.986950 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:59.987238 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:59.987635 (+ 397us) server_negotiation.cc:299] Negotiation successful
0619 02:55:59.988050 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":317,"threads_started":1}
I20260619 02:55:59.990597 9358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.975797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51620 (local address 127.18.134.62:35951)
0619 02:55:59.975987 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:59.975993 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:59.976722 (+ 729us) server_negotiation.cc:407] Connection header received
0619 02:55:59.976896 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:59.976901 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:59.976980 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:59.977088 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:59.978221 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.979288 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.980598 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.981240 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.981331 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:59.985268 (+ 3937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:55:59.985275 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:55:59.985280 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:55:59.985321 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:55:59.987897 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:59.988654 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:59.988658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:59.988660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:59.988803 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:55:59.989184 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:55:59.989187 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:55:59.989189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:55:59.989442 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:55:59.989593 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 02:55:59.990182 (+ 589us) server_negotiation.cc:299] Negotiation successful
0619 02:55:59.990351 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 02:55:59.990818 9507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.975758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.62:35951 (local address 127.0.0.1:51620)
0619 02:55:59.976570 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.976622 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:59.976785 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:59.977140 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:59.977144 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:59.977827 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:59.978060 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:59.978069 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.979447 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.979451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:59.980458 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:59.980473 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.981412 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.981418 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:59.981669 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:59.982399 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:55:59.982436 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:55:59.985082 (+ 2646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:55:59.988024 (+ 2942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:59.988034 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:59.988040 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:59.988539 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:59.988904 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:55:59.988908 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:55:59.988911 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:55:59.989090 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:55:59.989661 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:55:59.989667 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:55:59.989876 (+ 209us) client_negotiation.cc:769] Sending connection context
0619 02:55:59.990073 (+ 197us) client_negotiation.cc:240] Negotiation successful
0619 02:55:59.990386 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":385,"threads_started":1}
I20260619 02:55:59.992211 9503 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35733 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:55:59 dist-test-slave-9mr0 krb5kdc[9275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837759, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.60@KRBTEST.COM for kudu/127.18.134.61@KRBTEST.COM
Jun 19 02:55:59 dist-test-slave-9mr0 krb5kdc[9275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.62@KRBTEST.COM
I20260619 02:56:00.006172 9507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.992815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.61:35733 (local address 127.0.0.1:40336)
0619 02:55:59.993009 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 02:55:59.993024 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 02:55:59.993107 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:55:59.993378 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:55:59.993382 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:55:59.993725 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:55:59.993857 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:59.993864 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.994829 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.994834 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:59.995690 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:55:59.995701 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.996337 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.996343 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:55:59.996509 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:55:59.997064 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:55:59.997084 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:56:00.000025 (+ 2941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:56:00.003750 (+ 3725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:56:00.003755 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:56:00.003758 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:56:00.004160 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:56:00.004585 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:56:00.004588 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:56:00.004590 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:56:00.004727 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:56:00.005287 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:56:00.005292 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:56:00.005447 (+ 155us) client_negotiation.cc:769] Sending connection context
0619 02:56:00.005582 (+ 135us) client_negotiation.cc:240] Negotiation successful
0619 02:56:00.005856 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260619 02:56:00.006420 9433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.992956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40336 (local address 127.18.134.61:35733)
0619 02:55:59.993133 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:59.993139 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:59.993156 (+ 17us) server_negotiation.cc:407] Connection header received
0619 02:55:59.993212 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:59.993216 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:59.993271 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:59.993346 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:59.993991 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.994712 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.995811 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.996207 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.996322 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:56:00.000232 (+ 3910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:56:00.000239 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:56:00.000244 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:56:00.000285 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:56:00.003607 (+ 3322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:56:00.004308 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:56:00.004313 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:56:00.004316 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:56:00.004468 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:56:00.004830 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:56:00.004835 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:56:00.004839 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:56:00.005159 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:56:00.005307 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 02:56:00.005697 (+ 390us) server_negotiation.cc:299] Negotiation successful
0619 02:56:00.005931 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 02:56:00.006474 9358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:55:59.992450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51628 (local address 127.18.134.62:35951)
0619 02:55:59.992605 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 02:55:59.992610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:55:59.992626 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:55:59.992764 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:55:59.992768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:55:59.992836 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:55:59.992921 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:55:59.994004 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.994796 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.995921 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:55:59.996329 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:55:59.996433 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:56:00.000402 (+ 3969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:56:00.000406 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:56:00.000409 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:56:00.000442 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:56:00.003281 (+ 2839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:56:00.003974 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:56:00.003979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:56:00.003983 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:56:00.004126 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:56:00.004506 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:56:00.004511 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:56:00.004515 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:56:00.004865 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:56:00.005144 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0619 02:56:00.005958 (+ 814us) server_negotiation.cc:299] Negotiation successful
0619 02:56:00.006173 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 02:56:00.007485 9503 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 02:56:00 dist-test-slave-9mr0 krb5kdc[9275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837759, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.60@KRBTEST.COM for kudu/127.18.134.60@KRBTEST.COM
Jun 19 02:56:00 dist-test-slave-9mr0 krb5kdc[9275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.134.61@KRBTEST.COM
I20260619 02:56:00.019821 9507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:56:00.008105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:37421 (local address 127.0.0.1:43938)
0619 02:56:00.008246 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 02:56:00.008263 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 02:56:00.008344 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:56:00.008936 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:56:00.008939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:56:00.009256 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:56:00.009382 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:56:00.009389 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.010420 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.010425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:56:00.011394 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:56:00.011405 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.012222 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.012229 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:56:00.012426 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:56:00.013011 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:56:00.013027 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:56:00.014858 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:56:00.017651 (+ 2793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:56:00.017657 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:56:00.017661 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:56:00.017973 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:56:00.018282 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:56:00.018287 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:56:00.018291 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:56:00.018413 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:56:00.018914 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:56:00.018918 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:56:00.019048 (+ 130us) client_negotiation.cc:769] Sending connection context
0619 02:56:00.019184 (+ 136us) client_negotiation.cc:240] Negotiation successful
0619 02:56:00.019442 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 02:56:00.019893 9506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:56:00.008504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43938 (local address 127.18.134.60:37421)
0619 02:56:00.008684 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:00.008690 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:00.008706 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:56:00.008747 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:00.008752 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:00.008812 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:00.008902 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:56:00.009525 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.010302 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.011519 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.012077 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.012166 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:56:00.015021 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:56:00.015027 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:56:00.015032 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:56:00.015069 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:56:00.017530 (+ 2461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:56:00.018072 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:56:00.018077 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:56:00.018080 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:56:00.018179 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:56:00.018506 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:56:00.018511 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:56:00.018514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:56:00.018814 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:56:00.018936 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 02:56:00.019274 (+ 338us) server_negotiation.cc:299] Negotiation successful
0619 02:56:00.019600 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":119,"server-negotiator.queue_time_us":49}
I20260619 02:56:00.021415 9433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:56:00.008079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40346 (local address 127.18.134.61:35733)
0619 02:56:00.008225 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:00.008230 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:00.008246 (+ 16us) server_negotiation.cc:407] Connection header received
0619 02:56:00.008292 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:00.008297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:00.008359 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:00.008436 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:56:00.009842 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.010624 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.011916 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.012430 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.012547 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:56:00.016411 (+ 3864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:56:00.016416 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:56:00.016419 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:56:00.016446 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:56:00.018883 (+ 2437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:56:00.019457 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:56:00.019461 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:56:00.019463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:56:00.019586 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:56:00.020094 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:56:00.020099 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:56:00.020102 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:56:00.020391 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:56:00.020632 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0619 02:56:00.020897 (+ 265us) server_negotiation.cc:299] Negotiation successful
0619 02:56:00.021110 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 02:56:00.025298 9503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec2191bf86ab464da415dda416e87355: Bootstrap starting.
I20260619 02:56:00.026044 18968 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 02:56:00.029217 9503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec2191bf86ab464da415dda416e87355: Neither blocks nor log segments found. Creating new log.
I20260619 02:56:00.030471 9503 log.cc:824] T 00000000000000000000000000000000 P ec2191bf86ab464da415dda416e87355: Log is configured to *not* fsync() on all Append() calls
I20260619 02:56:00.028962 9330 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "0777f73666074a3fb84a979853d92ee4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51628
I20260619 02:56:00.033483 9503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec2191bf86ab464da415dda416e87355: No bootstrap required, opened a new log
I20260619 02:56:00.044039 9503 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec2191bf86ab464da415dda416e87355 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0777f73666074a3fb84a979853d92ee4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35951 } } peers { permanent_uuid: "f3e10f76d2294e418cc52e37c44dcf56" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35733 } } peers { permanent_uuid: "ec2191bf86ab464da415dda416e87355" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } }
I20260619 02:56:00.044464 9503 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec2191bf86ab464da415dda416e87355 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec2191bf86ab464da415dda416e87355, State: Initialized, Role: FOLLOWER
I20260619 02:56:00.044968 9503 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec2191bf86ab464da415dda416e87355 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0777f73666074a3fb84a979853d92ee4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35951 } } peers { permanent_uuid: "f3e10f76d2294e418cc52e37c44dcf56" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35733 } } peers { permanent_uuid: "ec2191bf86ab464da415dda416e87355" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } }
I20260619 02:56:00.045907 9509 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec2191bf86ab464da415dda416e87355 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0777f73666074a3fb84a979853d92ee4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35951 } } peers { permanent_uuid: "f3e10f76d2294e418cc52e37c44dcf56" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35733 } } peers { permanent_uuid: "ec2191bf86ab464da415dda416e87355" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } } }
I20260619 02:56:00.046231 9509 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec2191bf86ab464da415dda416e87355 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:56:00.047194 9503 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec2191bf86ab464da415dda416e87355 [sys.catalog]: configured and running, proceeding with master startup.
W20260619 02:56:00.056183 9520 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ec2191bf86ab464da415dda416e87355: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 02:56:00.056336 9520 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 02:56:00.054368 9436 master_runner.cc:186] Error getting master registration for 127.18.134.62:35951: OK, instance_id { permanent_uuid: "0777f73666074a3fb84a979853d92ee4" instance_seqno: 1781837758992176 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:56:00.062505 9436 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 02:56:00.124534 9355 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } attempt: 4
Jun 19 02:56:00 dist-test-slave-9mr0 krb5kdc[9275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837758, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.62@KRBTEST.COM for kudu/127.18.134.60@KRBTEST.COM
I20260619 02:56:00.140626 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:56:00.125350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:37421 (local address 127.0.0.1:43940)
0619 02:56:00.125535 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 02:56:00.125558 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 02:56:00.125696 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:56:00.126078 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:56:00.126082 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:56:00.126455 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:56:00.126624 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:56:00.126675 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.128092 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.128097 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:56:00.129581 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:56:00.129597 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.130650 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.130656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:56:00.130858 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:56:00.131522 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:56:00.131546 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:56:00.133793 (+ 2247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:56:00.137676 (+ 3883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:56:00.137682 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:56:00.137686 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:56:00.138195 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:56:00.138582 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:56:00.138587 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:56:00.138591 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:56:00.138794 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:56:00.139601 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:56:00.139606 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:56:00.139798 (+ 192us) client_negotiation.cc:769] Sending connection context
0619 02:56:00.139936 (+ 138us) client_negotiation.cc:240] Negotiation successful
0619 02:56:00.140226 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260619 02:56:00.140682 9506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:56:00.125435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43940 (local address 127.18.134.60:37421)
0619 02:56:00.125645 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:00.125651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:00.125685 (+ 34us) server_negotiation.cc:407] Connection header received
0619 02:56:00.125835 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:00.125840 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:00.125921 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:00.126043 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:56:00.126830 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.127951 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.129746 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.130482 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.130600 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:56:00.134016 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:56:00.134022 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:56:00.134028 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:56:00.134069 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:56:00.137518 (+ 3449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:56:00.138315 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:56:00.138321 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:56:00.138324 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:56:00.138465 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:56:00.138897 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:56:00.138902 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:56:00.138906 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:56:00.139485 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:56:00.139649 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 02:56:00.140081 (+ 432us) server_negotiation.cc:299] Negotiation successful
0619 02:56:00.140322 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 02:56:00.147755 9355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4: Bootstrap starting.
I20260619 02:56:00.151911 9355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4: Neither blocks nor log segments found. Creating new log.
I20260619 02:56:00.153221 9355 log.cc:824] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4: Log is configured to *not* fsync() on all Append() calls
I20260619 02:56:00.156788 9355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4: No bootstrap required, opened a new log
I20260619 02:56:00.164893 9355 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0777f73666074a3fb84a979853d92ee4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35951 } } peers { permanent_uuid: "f3e10f76d2294e418cc52e37c44dcf56" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35733 } } peers { permanent_uuid: "ec2191bf86ab464da415dda416e87355" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } }
I20260619 02:56:00.165310 9355 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0777f73666074a3fb84a979853d92ee4, State: Initialized, Role: FOLLOWER
I20260619 02:56:00.165836 9355 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0777f73666074a3fb84a979853d92ee4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35951 } } peers { permanent_uuid: "f3e10f76d2294e418cc52e37c44dcf56" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35733 } } peers { permanent_uuid: "ec2191bf86ab464da415dda416e87355" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } }
I20260619 02:56:00.166817 9523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0777f73666074a3fb84a979853d92ee4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35951 } } peers { permanent_uuid: "f3e10f76d2294e418cc52e37c44dcf56" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35733 } } peers { permanent_uuid: "ec2191bf86ab464da415dda416e87355" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } } }
I20260619 02:56:00.167200 9523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:56:00.168138 9355 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:56:00.168183 9330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 02:56:00.168349 9330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:56:00.170765 9330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0777f73666074a3fb84a979853d92ee4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35951 } } peers { permanent_uuid: "f3e10f76d2294e418cc52e37c44dcf56" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35733 } } peers { permanent_uuid: "ec2191bf86ab464da415dda416e87355" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } }
I20260619 02:56:00.172731 9478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0777f73666074a3fb84a979853d92ee4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ec2191bf86ab464da415dda416e87355"
I20260619 02:56:00.173131 9478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec2191bf86ab464da415dda416e87355 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:56:00.175776 9478 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ec2191bf86ab464da415dda416e87355 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0777f73666074a3fb84a979853d92ee4 in term 1.
I20260619 02:56:00.176474 9306 leader_election.cc:302] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0777f73666074a3fb84a979853d92ee4, ec2191bf86ab464da415dda416e87355; no voters:
I20260619 02:56:00.176605 9330 leader_election.cc:288] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4 [CANDIDATE]: Term 1 election: Requested vote from peers f3e10f76d2294e418cc52e37c44dcf56 (127.18.134.61:35733), ec2191bf86ab464da415dda416e87355 (127.18.134.60:37421)
I20260619 02:56:00.176973 9523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 02:56:00.189620 9429 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } attempt: 5
W20260619 02:56:00.194422 9457 tablet.cc:2369] T 00000000000000000000000000000000 P ec2191bf86ab464da415dda416e87355: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:56:00.197348 9523 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4 [term 1 LEADER]: Becoming Leader. State: Replica: 0777f73666074a3fb84a979853d92ee4, State: Running, Role: LEADER
Jun 19 02:56:00 dist-test-slave-9mr0 krb5kdc[9275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781837759, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.134.61@KRBTEST.COM for kudu/127.18.134.60@KRBTEST.COM
I20260619 02:56:00.207857 9431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:56:00.190340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.134.60:37421 (local address 127.0.0.1:43948)
0619 02:56:00.190818 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0619 02:56:00.190840 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 02:56:00.190970 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 02:56:00.191865 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 02:56:00.191870 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 02:56:00.192409 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0619 02:56:00.192629 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:56:00.192643 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.194057 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.194064 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:56:00.195483 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 02:56:00.195500 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.196729 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.196736 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 02:56:00.196970 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:56:00.197680 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 02:56:00.197707 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 02:56:00.200749 (+ 3042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 02:56:00.204980 (+ 4231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:56:00.204985 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:56:00.204990 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:56:00.205506 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:56:00.205968 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 02:56:00.205973 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 02:56:00.205976 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 02:56:00.206168 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 02:56:00.206871 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 02:56:00.206876 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 02:56:00.207054 (+ 178us) client_negotiation.cc:769] Sending connection context
0619 02:56:00.207197 (+ 143us) client_negotiation.cc:240] Negotiation successful
0619 02:56:00.207487 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334}
I20260619 02:56:00.208910 9506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:56:00.191210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43948 (local address 127.18.134.60:37421)
0619 02:56:00.191392 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:00.191398 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:00.191417 (+ 19us) server_negotiation.cc:407] Connection header received
0619 02:56:00.191478 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:00.191484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:00.191568 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:00.191663 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:56:00.192820 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.193877 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.195673 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.196368 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.196481 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:56:00.200987 (+ 4506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:56:00.200995 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:56:00.201000 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:56:00.201054 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:56:00.204813 (+ 3759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:56:00.205653 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:56:00.205659 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:56:00.205662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:56:00.205836 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:56:00.206300 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:56:00.206305 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:56:00.206308 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:56:00.206675 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:56:00.206811 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 02:56:00.208295 (+ 1484us) server_negotiation.cc:299] Negotiation successful
0619 02:56:00.208523 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 02:56:00.215580 9429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3e10f76d2294e418cc52e37c44dcf56: Bootstrap starting.
I20260619 02:56:00.220212 9429 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3e10f76d2294e418cc52e37c44dcf56: Neither blocks nor log segments found. Creating new log.
I20260619 02:56:00.220669 9523 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0777f73666074a3fb84a979853d92ee4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35951 } } peers { permanent_uuid: "f3e10f76d2294e418cc52e37c44dcf56" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35733 } } peers { permanent_uuid: "ec2191bf86ab464da415dda416e87355" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } }
I20260619 02:56:00.225006 9429 log.cc:824] T 00000000000000000000000000000000 P f3e10f76d2294e418cc52e37c44dcf56: Log is configured to *not* fsync() on all Append() calls
I20260619 02:56:00.228242 9528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0777f73666074a3fb84a979853d92ee4. Latest consensus state: current_term: 1 leader_uuid: "0777f73666074a3fb84a979853d92ee4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0777f73666074a3fb84a979853d92ee4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35951 } } peers { permanent_uuid: "f3e10f76d2294e418cc52e37c44dcf56" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35733 } } peers { permanent_uuid: "ec2191bf86ab464da415dda416e87355" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } } }
I20260619 02:56:00.228718 9528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4 [sys.catalog]: This master's current role is: LEADER
W20260619 02:56:00.232666 9309 tablet.cc:2369] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 02:56:00.236846 9544 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 02:56:00.241207 9288 master_runner.cc:186] Error getting master registration for 127.18.134.61:35733: OK, instance_id { permanent_uuid: "f3e10f76d2294e418cc52e37c44dcf56" instance_seqno: 1781837759469580 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 02:56:00.251430 9288 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 02:56:00.255654 9545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 02:56:00.263784 9429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3e10f76d2294e418cc52e37c44dcf56: No bootstrap required, opened a new log
I20260619 02:56:00.269094 9545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 02:56:00.292778 9358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:56:00.208212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51634 (local address 127.18.134.62:35951)
0619 02:56:00.208395 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:00.208400 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:00.254922 (+ 46522us) server_negotiation.cc:407] Connection header received
0619 02:56:00.255378 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:00.255384 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:00.255477 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:00.255564 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:56:00.260255 (+ 4691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.261321 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.269111 (+ 7790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.269818 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.269928 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:56:00.283033 (+ 13105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:56:00.283041 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:56:00.283046 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:56:00.283102 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:56:00.287009 (+ 3907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:56:00.288510 (+ 1501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:56:00.288516 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:56:00.288520 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:56:00.288829 (+ 309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:56:00.289569 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:56:00.289574 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:56:00.289577 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:56:00.290224 (+ 647us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:56:00.290704 (+ 480us) server_negotiation.cc:1035] Waiting for connection context
0619 02:56:00.291666 (+ 962us) server_negotiation.cc:299] Negotiation successful
0619 02:56:00.292104 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 02:56:00.279359 9429 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3e10f76d2294e418cc52e37c44dcf56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0777f73666074a3fb84a979853d92ee4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35951 } } peers { permanent_uuid: "f3e10f76d2294e418cc52e37c44dcf56" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35733 } } peers { permanent_uuid: "ec2191bf86ab464da415dda416e87355" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } }
I20260619 02:56:00.301164 9429 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3e10f76d2294e418cc52e37c44dcf56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e10f76d2294e418cc52e37c44dcf56, State: Initialized, Role: FOLLOWER
I20260619 02:56:00.301818 9429 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3e10f76d2294e418cc52e37c44dcf56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0777f73666074a3fb84a979853d92ee4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35951 } } peers { permanent_uuid: "f3e10f76d2294e418cc52e37c44dcf56" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35733 } } peers { permanent_uuid: "ec2191bf86ab464da415dda416e87355" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } }
I20260619 02:56:00.304234 9433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:56:00.258549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40356 (local address 127.18.134.61:35733)
0619 02:56:00.258764 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:00.258770 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:00.260767 (+ 1997us) server_negotiation.cc:407] Connection header received
0619 02:56:00.261165 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:00.261171 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:00.261245 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:00.261317 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:56:00.269538 (+ 8221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.270522 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.284363 (+ 13841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.285014 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.285140 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:56:00.292039 (+ 6899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:56:00.292046 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:56:00.292050 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:56:00.292107 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:56:00.295700 (+ 3593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:56:00.300962 (+ 5262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:56:00.300968 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:56:00.300971 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:56:00.301168 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:56:00.301777 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:56:00.301783 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:56:00.301786 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:56:00.302321 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:56:00.302570 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0619 02:56:00.303610 (+ 1040us) server_negotiation.cc:299] Negotiation successful
0619 02:56:00.303866 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260619 02:56:00.309274 9506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:56:00.215086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43950 (local address 127.18.134.60:37421)
0619 02:56:00.218926 (+ 3840us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:00.218932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:00.259597 (+ 40665us) server_negotiation.cc:407] Connection header received
0619 02:56:00.261568 (+ 1971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:00.261573 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:00.261684 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:00.261757 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:56:00.281146 (+ 19389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.282362 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.286092 (+ 3730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.286809 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.286912 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:56:00.296638 (+ 9726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 02:56:00.296650 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 02:56:00.296655 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 02:56:00.296702 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 02:56:00.300313 (+ 3611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:56:00.306428 (+ 6115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:56:00.306434 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:56:00.306437 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:56:00.306612 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 02:56:00.307112 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 02:56:00.307117 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 02:56:00.307120 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 02:56:00.307556 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 02:56:00.307809 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0619 02:56:00.308405 (+ 596us) server_negotiation.cc:299] Negotiation successful
0619 02:56:00.308820 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3696,"thread_start_us":39841,"threads_started":1}
I20260619 02:56:00.324908 9478 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ec2191bf86ab464da415dda416e87355 [term 1 FOLLOWER]: Refusing update from remote peer 0777f73666074a3fb84a979853d92ee4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:56:00.325233 9549 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3e10f76d2294e418cc52e37c44dcf56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0777f73666074a3fb84a979853d92ee4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35951 } } peers { permanent_uuid: "f3e10f76d2294e418cc52e37c44dcf56" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35733 } } peers { permanent_uuid: "ec2191bf86ab464da415dda416e87355" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } } }
I20260619 02:56:00.326094 9528 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec2191bf86ab464da415dda416e87355" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:56:00.326349 9429 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3e10f76d2294e418cc52e37c44dcf56 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 02:56:00.326745 9403 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3e10f76d2294e418cc52e37c44dcf56 [term 0 FOLLOWER]: Advancing to term 1
I20260619 02:56:00.327251 9404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0777f73666074a3fb84a979853d92ee4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f3e10f76d2294e418cc52e37c44dcf56"
I20260619 02:56:00.330470 9403 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f3e10f76d2294e418cc52e37c44dcf56 [term 1 FOLLOWER]: Refusing update from remote peer 0777f73666074a3fb84a979853d92ee4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 02:56:00.331413 9528 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3e10f76d2294e418cc52e37c44dcf56" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 02:56:00.332923 9549 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3e10f76d2294e418cc52e37c44dcf56 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:56:00.333946 9404 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f3e10f76d2294e418cc52e37c44dcf56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0777f73666074a3fb84a979853d92ee4 in term 1.
I20260619 02:56:00.373391 9509 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec2191bf86ab464da415dda416e87355 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0777f73666074a3fb84a979853d92ee4. Latest consensus state: current_term: 1 leader_uuid: "0777f73666074a3fb84a979853d92ee4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0777f73666074a3fb84a979853d92ee4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35951 } } peers { permanent_uuid: "f3e10f76d2294e418cc52e37c44dcf56" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35733 } } peers { permanent_uuid: "ec2191bf86ab464da415dda416e87355" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } } }
I20260619 02:56:00.373745 9509 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec2191bf86ab464da415dda416e87355 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:56:00.377658 9549 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3e10f76d2294e418cc52e37c44dcf56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0777f73666074a3fb84a979853d92ee4. Latest consensus state: current_term: 1 leader_uuid: "0777f73666074a3fb84a979853d92ee4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0777f73666074a3fb84a979853d92ee4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35951 } } peers { permanent_uuid: "f3e10f76d2294e418cc52e37c44dcf56" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35733 } } peers { permanent_uuid: "ec2191bf86ab464da415dda416e87355" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } } }
I20260619 02:56:00.377988 9549 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3e10f76d2294e418cc52e37c44dcf56 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:56:00.380721 9548 mvcc.cc:205] Tried to move back new op lower bound from 7298407466227941376 to 7298407465877196800. Current Snapshot: MvccSnapshot[applied={T|T < 7298407466227941376}]
I20260619 02:56:00.393041 9549 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3e10f76d2294e418cc52e37c44dcf56 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0777f73666074a3fb84a979853d92ee4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0777f73666074a3fb84a979853d92ee4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35951 } } peers { permanent_uuid: "f3e10f76d2294e418cc52e37c44dcf56" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35733 } } peers { permanent_uuid: "ec2191bf86ab464da415dda416e87355" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } } }
I20260619 02:56:00.393364 9549 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3e10f76d2294e418cc52e37c44dcf56 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:56:00.395153 9509 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec2191bf86ab464da415dda416e87355 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0777f73666074a3fb84a979853d92ee4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0777f73666074a3fb84a979853d92ee4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35951 } } peers { permanent_uuid: "f3e10f76d2294e418cc52e37c44dcf56" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35733 } } peers { permanent_uuid: "ec2191bf86ab464da415dda416e87355" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } } }
I20260619 02:56:00.395418 9509 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec2191bf86ab464da415dda416e87355 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 02:56:00.396721 9528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0777f73666074a3fb84a979853d92ee4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0777f73666074a3fb84a979853d92ee4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35951 } } peers { permanent_uuid: "f3e10f76d2294e418cc52e37c44dcf56" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35733 } } peers { permanent_uuid: "ec2191bf86ab464da415dda416e87355" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } } }
I20260619 02:56:00.397003 9528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4 [sys.catalog]: This master's current role is: LEADER
I20260619 02:56:00.397197 9528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0777f73666074a3fb84a979853d92ee4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0777f73666074a3fb84a979853d92ee4" member_type: VOTER last_known_addr { host: "127.18.134.62" port: 35951 } } peers { permanent_uuid: "f3e10f76d2294e418cc52e37c44dcf56" member_type: VOTER last_known_addr { host: "127.18.134.61" port: 35733 } } peers { permanent_uuid: "ec2191bf86ab464da415dda416e87355" member_type: VOTER last_known_addr { host: "127.18.134.60" port: 37421 } } }
I20260619 02:56:00.397400 9528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4 [sys.catalog]: This master's current role is: LEADER
I20260619 02:56:00.398168 9545 catalog_manager.cc:1347] Generated new cluster ID: 6ddae0c6715c4375a1ce759e05c31b4f
I20260619 02:56:00.398270 9545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 02:56:00.435086 9545 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 02:56:00.436908 9545 catalog_manager.cc:1504] Loading token signing keys...
I20260619 02:56:00.454847 9545 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0777f73666074a3fb84a979853d92ee4: Generated new TSK 0
I20260619 02:56:00.455920 9545 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 02:56:00.463179 9568 catalog_manager.cc:1259] Loaded cluster ID: 6ddae0c6715c4375a1ce759e05c31b4f
I20260619 02:56:00.463383 9568 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f3e10f76d2294e418cc52e37c44dcf56: loading cluster ID for follower catalog manager: success
I20260619 02:56:00.487915 9568 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f3e10f76d2294e418cc52e37c44dcf56: acquiring CA information for follower catalog manager: success
I20260619 02:56:00.491462 9568 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f3e10f76d2294e418cc52e37c44dcf56: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 02:56:00.585692 9358 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:56:00.582746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51646 (local address 127.18.134.62:35951)
0619 02:56:00.582941 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:00.582947 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:00.583427 (+ 480us) server_negotiation.cc:407] Connection header received
0619 02:56:00.583719 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:00.583725 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:00.583865 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:00.583953 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:56:00.585411 (+ 1458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51646: BlockingRecv error: recv got EOF from 127.0.0.1:51646 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260619 02:56:00.588253 9506 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:56:00.583177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43962 (local address 127.18.134.60:37421)
0619 02:56:00.583346 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:00.583351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:00.586518 (+ 3167us) server_negotiation.cc:407] Connection header received
0619 02:56:00.586790 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:00.586796 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:00.586889 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:00.586959 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:56:00.587987 (+ 1028us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43962: BlockingRecv error: recv got EOF from 127.0.0.1:43962 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260619 02:56:00.590662 9433 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:56:00.583770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40366 (local address 127.18.134.61:35733)
0619 02:56:00.584279 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:00.584286 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:00.588944 (+ 4658us) server_negotiation.cc:407] Connection header received
0619 02:56:00.589159 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:00.589166 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:00.589257 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:00.589335 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:56:00.590318 (+ 983us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40366: BlockingRecv error: recv got EOF from 127.0.0.1:40366 (error 108)
Metrics: {"server-negotiator.queue_time_us":319}
I20260619 02:56:00.610355 9433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:56:00.599444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40378 (local address 127.18.134.61:35733)
0619 02:56:00.599619 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:00.599626 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:00.603170 (+ 3544us) server_negotiation.cc:407] Connection header received
0619 02:56:00.603440 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:00.603447 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:00.603672 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:00.603848 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:56:00.604309 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.605463 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.607409 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.608095 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.608226 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:56:00.608834 (+ 608us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:56:00.609539 (+ 705us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:56:00.609655 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 02:56:00.609895 (+ 240us) server_negotiation.cc:299] Negotiation successful
0619 02:56:00.610013 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260619 02:56:00.619755 9506 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:56:00.617738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43968 (local address 127.18.134.60:37421)
0619 02:56:00.617922 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:00.617929 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:00.618045 (+ 116us) server_negotiation.cc:407] Connection header received
0619 02:56:00.618372 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:00.618378 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:00.618506 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:00.618610 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:56:00.619510 (+ 900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43968: BlockingRecv error: recv got EOF from 127.0.0.1:43968 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 02:56:00.633299 9433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:56:00.624578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40394 (local address 127.18.134.61:35733)
0619 02:56:00.624776 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:00.624782 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:00.627139 (+ 2357us) server_negotiation.cc:407] Connection header received
0619 02:56:00.627312 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:00.627318 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:00.627434 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:00.627531 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:56:00.627997 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.629148 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.631107 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.631818 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.631949 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:56:00.632515 (+ 566us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:56:00.632670 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:56:00.632763 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 02:56:00.632909 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 02:56:00.632991 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260619 02:56:00.643049 9506 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:56:00.640929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43970 (local address 127.18.134.60:37421)
0619 02:56:00.641112 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:00.641118 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:00.641576 (+ 458us) server_negotiation.cc:407] Connection header received
0619 02:56:00.641781 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:00.641788 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:00.641898 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:00.642009 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:56:00.642831 (+ 822us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43970: BlockingRecv error: recv got EOF from 127.0.0.1:43970 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:56:00.655400 9433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:56:00.648374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40398 (local address 127.18.134.61:35733)
0619 02:56:00.648551 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:00.648557 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:00.649074 (+ 517us) server_negotiation.cc:407] Connection header received
0619 02:56:00.649266 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:00.649272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:00.649356 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:00.649443 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:56:00.649895 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.651109 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.653089 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.653753 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.653876 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:56:00.654356 (+ 480us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:56:00.654519 (+ 163us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:56:00.654697 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0619 02:56:00.654966 (+ 269us) server_negotiation.cc:299] Negotiation successful
0619 02:56:00.655056 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260619 02:56:00.664450 9506 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:56:00.662244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43974 (local address 127.18.134.60:37421)
0619 02:56:00.662425 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:00.662431 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:00.662944 (+ 513us) server_negotiation.cc:407] Connection header received
0619 02:56:00.663134 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:00.663140 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:00.663251 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:00.663352 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:56:00.664193 (+ 841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43974: BlockingRecv error: recv got EOF from 127.0.0.1:43974 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 02:56:00.678719 9433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:56:00.671735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40410 (local address 127.18.134.61:35733)
0619 02:56:00.671971 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:00.671977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:00.672505 (+ 528us) server_negotiation.cc:407] Connection header received
0619 02:56:00.672687 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:00.672693 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:00.672806 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:00.672950 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:56:00.673374 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.674547 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.676550 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.677269 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.677405 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:56:00.677880 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:56:00.678030 (+ 150us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:56:00.678122 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 02:56:00.678268 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 02:56:00.678357 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
W20260619 02:56:00.691165 9506 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:56:00.689135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43988 (local address 127.18.134.60:37421)
0619 02:56:00.689387 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:00.689393 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:00.689416 (+ 23us) server_negotiation.cc:407] Connection header received
0619 02:56:00.689628 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:00.689635 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:00.689736 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:00.689811 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:56:00.690929 (+ 1118us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43988: BlockingRecv error: recv got EOF from 127.0.0.1:43988 (error 108)
Metrics: {"server-negotiator.queue_time_us":136}
I20260619 02:56:00.710533 9433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:56:00.703518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40426 (local address 127.18.134.61:35733)
0619 02:56:00.703717 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:00.703724 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:00.704256 (+ 532us) server_negotiation.cc:407] Connection header received
0619 02:56:00.704501 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:00.704508 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:00.704601 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:00.704712 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:56:00.705195 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.706350 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.708326 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.709046 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.709185 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:56:00.709702 (+ 517us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:56:00.709859 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:56:00.709943 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 02:56:00.710107 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 02:56:00.710194 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260619 02:56:00.720139 9506 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:56:00.717502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43998 (local address 127.18.134.60:37421)
0619 02:56:00.717676 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:00.717681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:00.718224 (+ 543us) server_negotiation.cc:407] Connection header received
0619 02:56:00.718410 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:00.718417 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:00.718520 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:00.718658 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:56:00.719885 (+ 1227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43998: BlockingRecv error: recv got EOF from 127.0.0.1:43998 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 02:56:00.747603 9433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:56:00.740259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40430 (local address 127.18.134.61:35733)
0619 02:56:00.740483 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:00.740490 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:00.741071 (+ 581us) server_negotiation.cc:407] Connection header received
0619 02:56:00.741323 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:00.741329 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:00.741428 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:00.741532 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:56:00.742052 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.743340 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.745356 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.746037 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.746182 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:56:00.746757 (+ 575us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:56:00.746901 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:56:00.746996 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 02:56:00.747205 (+ 209us) server_negotiation.cc:299] Negotiation successful
0619 02:56:00.747317 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
W20260619 02:56:00.758770 9506 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:56:00.756471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44014 (local address 127.18.134.60:37421)
0619 02:56:00.756663 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:00.756668 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:00.757198 (+ 530us) server_negotiation.cc:407] Connection header received
0619 02:56:00.757426 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:00.757432 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:00.757535 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:00.757647 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:56:00.758490 (+ 843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44014: BlockingRecv error: recv got EOF from 127.0.0.1:44014 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 02:56:00.808688 9433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:56:00.801170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40434 (local address 127.18.134.61:35733)
0619 02:56:00.801405 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:00.801412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:00.801932 (+ 520us) server_negotiation.cc:407] Connection header received
0619 02:56:00.802302 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:00.802308 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:00.802399 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:00.802489 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:56:00.803194 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.804353 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.806328 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.807028 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.807133 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:56:00.807792 (+ 659us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:56:00.807935 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:56:00.808004 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 02:56:00.808260 (+ 256us) server_negotiation.cc:299] Negotiation successful
0619 02:56:00.808351 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
W20260619 02:56:00.818536 9506 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:56:00.816123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44026 (local address 127.18.134.60:37421)
0619 02:56:00.816350 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:00.816357 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:00.816908 (+ 551us) server_negotiation.cc:407] Connection header received
0619 02:56:00.817127 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:00.817133 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:00.817247 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:00.817355 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:56:00.818277 (+ 922us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44026: BlockingRecv error: recv got EOF from 127.0.0.1:44026 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 02:56:00.892158 9433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:56:00.886082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40448 (local address 127.18.134.61:35733)
0619 02:56:00.886276 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:00.886283 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:00.886648 (+ 365us) server_negotiation.cc:407] Connection header received
0619 02:56:00.886853 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:00.886858 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:00.886943 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:00.887043 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:56:00.887436 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.888621 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.890093 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:00.890802 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:00.890925 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:56:00.891301 (+ 376us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:56:00.891443 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:56:00.891537 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 02:56:00.891746 (+ 209us) server_negotiation.cc:299] Negotiation successful
0619 02:56:00.891875 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260619 02:56:00.900924 9506 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:56:00.898939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44030 (local address 127.18.134.60:37421)
0619 02:56:00.899132 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:00.899137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:00.899498 (+ 361us) server_negotiation.cc:407] Connection header received
0619 02:56:00.899668 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:00.899672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:00.899748 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:00.899828 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:56:00.900715 (+ 887us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44030: BlockingRecv error: recv got EOF from 127.0.0.1:44030 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 02:56:01.037591 9433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:56:01.032689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40452 (local address 127.18.134.61:35733)
0619 02:56:01.032919 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:01.032925 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:01.033209 (+ 284us) server_negotiation.cc:407] Connection header received
0619 02:56:01.033411 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:01.033415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:01.033485 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:01.033556 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:56:01.033989 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:01.034703 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:01.036009 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:01.036442 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:01.036549 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:56:01.036946 (+ 397us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:56:01.037049 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:56:01.037121 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 02:56:01.037264 (+ 143us) server_negotiation.cc:299] Negotiation successful
0619 02:56:01.037345 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260619 02:56:01.044576 9506 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 02:56:01.042795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44036 (local address 127.18.134.60:37421)
0619 02:56:01.042994 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:01.043000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:01.043255 (+ 255us) server_negotiation.cc:407] Connection header received
0619 02:56:01.043428 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:01.043434 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:01.043514 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:01.043596 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 02:56:01.044341 (+ 745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44036: BlockingRecv error: recv got EOF from 127.0.0.1:44036 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 02:56:01.058364 9520 catalog_manager.cc:1259] Loaded cluster ID: 6ddae0c6715c4375a1ce759e05c31b4f
I20260619 02:56:01.058476 9520 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ec2191bf86ab464da415dda416e87355: loading cluster ID for follower catalog manager: success
I20260619 02:56:01.061921 9520 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ec2191bf86ab464da415dda416e87355: acquiring CA information for follower catalog manager: success
I20260619 02:56:01.063916 9520 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ec2191bf86ab464da415dda416e87355: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 02:56:01.312166 9433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:56:01.307211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40462 (local address 127.18.134.61:35733)
0619 02:56:01.307426 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:01.307433 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:01.307648 (+ 215us) server_negotiation.cc:407] Connection header received
0619 02:56:01.307856 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:01.307861 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:01.307931 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:01.308003 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:56:01.308417 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:01.309210 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:01.310532 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:01.310995 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:01.311110 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:56:01.311488 (+ 378us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:56:01.311614 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:56:01.311690 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 02:56:01.311830 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 02:56:01.311912 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 02:56:01.324412 9506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:56:01.318539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44050 (local address 127.18.134.60:37421)
0619 02:56:01.318778 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:01.318784 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:01.318867 (+ 83us) server_negotiation.cc:407] Connection header received
0619 02:56:01.319066 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:01.319072 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:01.319236 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:01.319338 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:56:01.319778 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:01.320490 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:01.322192 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:01.322614 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:01.322763 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:56:01.323283 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:56:01.323828 (+ 545us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:56:01.323963 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 02:56:01.324050 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 02:56:01.324145 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260619 02:56:01.337772 9506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:56:01.330876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44052 (local address 127.18.134.60:37421)
0619 02:56:01.331070 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:01.331077 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:01.331973 (+ 896us) server_negotiation.cc:407] Connection header received
0619 02:56:01.332259 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:01.332266 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:01.332385 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:01.332487 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:56:01.333142 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:01.334254 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:01.335956 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:01.336413 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:01.336524 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:56:01.336974 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:56:01.337144 (+ 170us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:56:01.337231 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 02:56:01.337375 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 02:56:01.337458 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 02:56:01.338809 9433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:56:01.331935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40472 (local address 127.18.134.61:35733)
0619 02:56:01.332151 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:01.332157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:01.332700 (+ 543us) server_negotiation.cc:407] Connection header received
0619 02:56:01.332888 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:01.332893 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:01.332977 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:01.333075 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:56:01.333619 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:01.334381 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:01.336870 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:01.337325 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:01.337434 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:56:01.338011 (+ 577us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:56:01.338139 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:56:01.338187 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 02:56:01.338406 (+ 219us) server_negotiation.cc:299] Negotiation successful
0619 02:56:01.338490 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260619 02:56:01.339301 9685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 02:56:01.330883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51652 (local address 127.18.134.62:35951)
0619 02:56:01.331390 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0619 02:56:01.331396 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 02:56:01.331915 (+ 519us) server_negotiation.cc:407] Connection header received
0619 02:56:01.332280 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 02:56:01.332287 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 02:56:01.332519 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 02:56:01.332683 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 02:56:01.333416 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:01.334431 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:01.336848 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 02:56:01.337400 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 02:56:01.337517 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 02:56:01.338230 (+ 713us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 02:56:01.338726 (+ 496us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 02:56:01.338833 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 02:56:01.338937 (+ 104us) server_negotiation.cc:299] Negotiation successful
0619 02:56:01.339026 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":294,"threads_started":1}
I20260619 02:56:01.344363 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 9288
I20260619 02:56:01.345386 9350 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:56:01.455397 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 9288
I20260619 02:56:01.472218 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 9362
I20260619 02:56:01.473346 9424 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:56:01.578846 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 9362
I20260619 02:56:01.595120 18968 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu pid 9436
I20260619 02:56:01.596297 9498 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 02:56:01.700677 18968 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskUJ45gP/build/asan/bin/kudu with pid 9436
2026-06-19T02:56:01Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4387 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (8551 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (351144 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST